[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:34:54 dist-test-slave-jq4t krb5kdc[1930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:34:54 dist-test-slave-jq4t krb5kdc[1930](info): set up 2 sockets
Aug 09 03:34:54 dist-test-slave-jq4t krb5kdc[1930](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 03:34:55.777830 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:34:55 dist-test-slave-jq4t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:34:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:34:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:34:55.910341 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:38431
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:38431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:56.043928 1951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:56.044292 1951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:56.044384 1951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:56.048568 1951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:34:56.048673 1951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:56.048712 1951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:56.048751 1951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:34:56.048786 1951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:34:56.054788 1951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:38431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:38431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:34:56.056527 1951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:56.057976 1951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:56.071400 1963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:56.073689 1965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:56.074045 1961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:56.074266 1951 server_base.cc:1034] running on GCE node
I20260809 03:34:56.075301 1951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:56.077366 1951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:56.078600 1951 hybrid_clock.cc:648] HybridClock initialized: now 1786246496078545 us; error 95 us; skew 500 ppm
Aug 09 03:34:56 dist-test-slave-jq4t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:56.081900 1951 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:34:56.083140 1951 webserver.cc:460] Webserver started at http://127.1.224.126:37803/ using document root <none> and password file <none>
I20260809 03:34:56.084126 1951 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:56.084235 1951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:56.084743 1951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:34:56.087575 1951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "4978832850484e15b29be26af99af35d"
format_stamp: "Formatted at 2026-08-09 03:34:56 on dist-test-slave-jq4t"
server_key: "fe1167d854d73fd06a71735866fd254a"
server_key_iv: "a618a3db3b57bf540eaf8da59b927318"
server_key_version: "encryptionkey@0"
I20260809 03:34:56.088222 1951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "4978832850484e15b29be26af99af35d"
format_stamp: "Formatted at 2026-08-09 03:34:56 on dist-test-slave-jq4t"
server_key: "fe1167d854d73fd06a71735866fd254a"
server_key_iv: "a618a3db3b57bf540eaf8da59b927318"
server_key_version: "encryptionkey@0"
I20260809 03:34:56.093750 1951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 03:34:56.097585 1975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:56.099399 1951 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:34:56.099629 1951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "4978832850484e15b29be26af99af35d"
format_stamp: "Formatted at 2026-08-09 03:34:56 on dist-test-slave-jq4t"
server_key: "fe1167d854d73fd06a71735866fd254a"
server_key_iv: "a618a3db3b57bf540eaf8da59b927318"
server_key_version: "encryptionkey@0"
I20260809 03:34:56.099767 1951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:56.114868 1951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:56.119329 1951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:56.119614 1951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:56.135195 1951 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:38431
I20260809 03:34:56.135314 2060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:38431 every 8 connection(s)
I20260809 03:34:56.136729 1951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:34:56.139539 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 1951
I20260809 03:34:56.139688 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:34:56.141278 1921 external_mini_cluster.cc:1428] Setting key d43b4df27efd15fa405b59724cd70f60
I20260809 03:34:56.142118 2061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:34:56.151916 2061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d: Bootstrap starting.
I20260809 03:34:56.155750 2061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d: Neither blocks nor log segments found. Creating new log.
I20260809 03:34:56.157049 2061 log.cc:824] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:34:56.159823 2061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d: No bootstrap required, opened a new log
Aug 09 03:34:56 dist-test-slave-jq4t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:34:56.164971 2061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978832850484e15b29be26af99af35d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 38431 } }
I20260809 03:34:56.165297 2061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:34:56.165409 2061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4978832850484e15b29be26af99af35d, State: Initialized, Role: FOLLOWER
I20260809 03:34:56.166265 2061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978832850484e15b29be26af99af35d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 38431 } }
I20260809 03:34:56.166507 2061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:34:56.166661 2061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:34:56.166826 2061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:34:56.168125 2061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978832850484e15b29be26af99af35d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 38431 } }
I20260809 03:34:56.168938 2061 leader_election.cc:302] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4978832850484e15b29be26af99af35d; no voters:
I20260809 03:34:56.169366 2061 leader_election.cc:288] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:34:56.169430 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:56.146162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54164 (local address 127.1.224.126:38431)
0809 03:34:56.146663 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:56.146680 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:56.147827 (+ 1147us) server_negotiation.cc:407] Connection header received
0809 03:34:56.151179 (+ 3352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:56.151196 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:56.151581 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:56.151909 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:56.154338 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.155408 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.156735 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.157224 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.157417 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:56.162295 (+ 4878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:56.162333 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:56.162346 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:56.162375 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:56.165234 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:56.165902 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:56.165909 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:56.165914 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:56.166038 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:56.166404 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:56.166407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:56.166409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:56.167095 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:56.167327 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:56.167956 (+ 629us) server_negotiation.cc:299] Negotiation successful
0809 03:34:56.168250 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":160,"threads_started":1}
I20260809 03:34:56.169528 2068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:34:56.169785 2068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [term 1 LEADER]: Becoming Leader. State: Replica: 4978832850484e15b29be26af99af35d, State: Running, Role: LEADER
I20260809 03:34:56.170264 2068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978832850484e15b29be26af99af35d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 38431 } }
I20260809 03:34:56.171239 2061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:34:56.174275 2070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4978832850484e15b29be26af99af35d. Latest consensus state: current_term: 1 leader_uuid: "4978832850484e15b29be26af99af35d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978832850484e15b29be26af99af35d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 38431 } } }
I20260809 03:34:56.174422 2070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [sys.catalog]: This master's current role is: LEADER
I20260809 03:34:56.175357 2069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4978832850484e15b29be26af99af35d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4978832850484e15b29be26af99af35d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 38431 } } }
I20260809 03:34:56.175495 2069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d [sys.catalog]: This master's current role is: LEADER
W20260809 03:34:56.179230 2087 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:34:56.179381 2087 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:34:56.179894 2082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:34:56.181013 2082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:34:56.189890 2082 catalog_manager.cc:1347] Generated new cluster ID: e6480bc027264c039c28621b4892548a
I20260809 03:34:56.189989 2082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:34:56.201364 2082 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:34:56.202862 2082 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:34:56.213946 2082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4978832850484e15b29be26af99af35d: Generated new TSK 0
I20260809 03:34:56.214872 2082 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:34:56.257616 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:38431
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:56.446187 2096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:56.446480 2096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:56.446568 2096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:56.450696 2096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:56.450765 2096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:56.450924 2096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:34:56.456825 2096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:38431
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:34:56.458087 2096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:56.459208 2096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:56.470662 2104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:56.470702 2111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:56.470788 2096 server_base.cc:1034] running on GCE node
W20260809 03:34:56.470724 2109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:56.471563 2096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:56.472312 2096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:56.473520 2096 hybrid_clock.cc:648] HybridClock initialized: now 1786246496473495 us; error 79 us; skew 500 ppm
Aug 09 03:34:56 dist-test-slave-jq4t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:56.476639 2096 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:34:56.477685 2096 webserver.cc:460] Webserver started at http://127.1.224.65:36591/ using document root <none> and password file <none>
I20260809 03:34:56.478328 2096 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:56.478427 2096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:56.478686 2096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:34:56.480978 2096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "a8d98677e7b346fd853f7edd1c00101f"
format_stamp: "Formatted at 2026-08-09 03:34:56 on dist-test-slave-jq4t"
server_key: "ca2da7ee3cdd1ccdf22e09136aa81b3e"
server_key_iv: "00f9ac40eb0ed42ed29bb32a95451914"
server_key_version: "encryptionkey@0"
I20260809 03:34:56.481602 2096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "a8d98677e7b346fd853f7edd1c00101f"
format_stamp: "Formatted at 2026-08-09 03:34:56 on dist-test-slave-jq4t"
server_key: "ca2da7ee3cdd1ccdf22e09136aa81b3e"
server_key_iv: "00f9ac40eb0ed42ed29bb32a95451914"
server_key_version: "encryptionkey@0"
I20260809 03:34:56.485615 2096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:34:56.488162 2119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:56.489440 2096 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:34:56.489600 2096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "a8d98677e7b346fd853f7edd1c00101f"
format_stamp: "Formatted at 2026-08-09 03:34:56 on dist-test-slave-jq4t"
server_key: "ca2da7ee3cdd1ccdf22e09136aa81b3e"
server_key_iv: "00f9ac40eb0ed42ed29bb32a95451914"
server_key_version: "encryptionkey@0"
I20260809 03:34:56.489725 2096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:56.502067 2096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:56.505923 2096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:56.506189 2096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:56.506989 2096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:56.508288 2096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:56.508373 2096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:56.508451 2096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:56.508541 2096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 03:34:56.518803 2096 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:42281
I20260809 03:34:56.518826 2326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:42281 every 8 connection(s)
I20260809 03:34:56.519797 2096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:34:56.527943 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 2096
I20260809 03:34:56.528083 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:34:56.528412 1921 external_mini_cluster.cc:1428] Setting key e0078dc416f736e7d804233940823114
Aug 09 03:34:56 dist-test-slave-jq4t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:34:56.535871 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:56.522165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:40343 (local address 127.1.224.126:38431)
0809 03:34:56.522314 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:56.522318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:56.523187 (+ 869us) server_negotiation.cc:407] Connection header received
0809 03:34:56.524798 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:56.524801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:56.524858 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:56.524940 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:56.526266 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.526831 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.528314 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.528651 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.528748 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:56.531659 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:56.531667 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:56.531673 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:56.531711 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:56.533367 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:56.534077 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:56.534079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:56.534081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:56.534158 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:56.534571 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:56.534573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:56.534574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:56.534759 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:56.534837 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:56.535539 (+ 702us) server_negotiation.cc:299] Negotiation successful
0809 03:34:56.535681 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:34:56.537249 2336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:56.522508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:38431 (local address 127.1.224.65:40343)
0809 03:34:56.522998 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:56.523056 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:56.524586 (+ 1530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:56.525129 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:56.525135 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:56.525550 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:56.526076 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:56.526092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.526974 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.526980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:56.528165 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:56.528180 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.528837 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.528841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:56.529090 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:56.529770 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:56.529791 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:56.531457 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:56.533499 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:56.533508 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:56.533528 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:56.533963 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:56.534250 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:56.534256 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:56.534260 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:56.534473 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:56.534868 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:56.534878 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:56.535247 (+ 369us) client_negotiation.cc:769] Sending connection context
0809 03:34:56.535510 (+ 263us) client_negotiation.cc:240] Negotiation successful
0809 03:34:56.535940 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":122,"threads_started":1}
I20260809 03:34:56.538697 2327 heartbeater.cc:347] Connected to a master server at 127.1.224.126:38431
I20260809 03:34:56.538976 2327 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:56.539521 2327 heartbeater.cc:511] Master 127.1.224.126:38431 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:34:56.541909 2001 ts_manager.cc:194] Registered new tserver with Master: a8d98677e7b346fd853f7edd1c00101f (127.1.224.65:42281)
I20260809 03:34:56.543526 2001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:40343
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:34:56.563207 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:38431
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:56.712815 2341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:56.713167 2341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:56.713269 2341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:56.717695 2341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:56.717788 2341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:56.717880 2341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:34:56.723453 2341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:38431
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:34:56.724812 2341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:56.725921 2341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:56.736307 2350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:56.736364 2348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:56.736302 2352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:56.736567 2341 server_base.cc:1034] running on GCE node
I20260809 03:34:56.737069 2341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:56.737679 2341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:56.738834 2341 hybrid_clock.cc:648] HybridClock initialized: now 1786246496738803 us; error 46 us; skew 500 ppm
Aug 09 03:34:56 dist-test-slave-jq4t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:56.741881 2341 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:34:56.742861 2341 webserver.cc:460] Webserver started at http://127.1.224.66:34193/ using document root <none> and password file <none>
I20260809 03:34:56.743467 2341 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:56.743559 2341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:56.743805 2341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:34:56.745963 2341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "040abea54eb14cd5a76bbf26eff15a3d"
format_stamp: "Formatted at 2026-08-09 03:34:56 on dist-test-slave-jq4t"
server_key: "62faf08318b13efe9613456fe201cebd"
server_key_iv: "9f533918d0c178e0c84fa11af4620380"
server_key_version: "encryptionkey@0"
I20260809 03:34:56.746564 2341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "040abea54eb14cd5a76bbf26eff15a3d"
format_stamp: "Formatted at 2026-08-09 03:34:56 on dist-test-slave-jq4t"
server_key: "62faf08318b13efe9613456fe201cebd"
server_key_iv: "9f533918d0c178e0c84fa11af4620380"
server_key_version: "encryptionkey@0"
I20260809 03:34:56.750557 2341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:34:56.753262 2361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:56.754549 2341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:34:56.754695 2341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "040abea54eb14cd5a76bbf26eff15a3d"
format_stamp: "Formatted at 2026-08-09 03:34:56 on dist-test-slave-jq4t"
server_key: "62faf08318b13efe9613456fe201cebd"
server_key_iv: "9f533918d0c178e0c84fa11af4620380"
server_key_version: "encryptionkey@0"
I20260809 03:34:56.754819 2341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:56.775677 2341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:56.779248 2341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:56.779469 2341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:56.780124 2341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:56.781117 2341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:56.781196 2341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:56.781270 2341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:56.781307 2341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:56.792340 2341 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:39661
I20260809 03:34:56.792364 2548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:39661 every 8 connection(s)
I20260809 03:34:56.793416 2341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:34:56.802095 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 2341
I20260809 03:34:56.802232 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:34:56.802506 1921 external_mini_cluster.cc:1428] Setting key 48d0daa9329b14d4bc396f45c82be497
Aug 09 03:34:56 dist-test-slave-jq4t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:34:56.809392 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:56.795630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:35463 (local address 127.1.224.126:38431)
0809 03:34:56.795807 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:56.795811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:56.797357 (+ 1546us) server_negotiation.cc:407] Connection header received
0809 03:34:56.798349 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:56.798354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:56.798419 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:56.798534 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:56.799826 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.800415 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.801666 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.801926 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.802051 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:56.805141 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:56.805145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:56.805147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:56.805172 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:56.806972 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:56.807509 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:56.807512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:56.807514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:56.807617 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:56.807965 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:56.807971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:56.807976 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:56.808253 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:56.808368 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:56.808964 (+ 596us) server_negotiation.cc:299] Negotiation successful
0809 03:34:56.809122 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 03:34:56.810210 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:56.795983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:38431 (local address 127.1.224.66:35463)
0809 03:34:56.797164 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:56.797214 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:56.798115 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:56.798731 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:56.798738 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:56.799127 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:56.799652 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:56.799667 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.800591 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.800594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:56.801528 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:56.801539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:56.802116 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:56.802120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:56.802363 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:56.803099 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:56.803121 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:56.804865 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:56.807102 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:56.807106 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:56.807121 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:56.807391 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:56.807720 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:56.807722 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:56.807723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:56.807858 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:56.808364 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:56.808369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:56.808697 (+ 328us) client_negotiation.cc:769] Sending connection context
0809 03:34:56.808867 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 03:34:56.809151 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"thread_start_us":752,"threads_started":1}
I20260809 03:34:56.811573 2550 heartbeater.cc:347] Connected to a master server at 127.1.224.126:38431
I20260809 03:34:56.811872 2550 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:56.812584 2550 heartbeater.cc:511] Master 127.1.224.126:38431 requested a full tablet report, sending...
I20260809 03:34:56.813942 2001 ts_manager.cc:194] Registered new tserver with Master: 040abea54eb14cd5a76bbf26eff15a3d (127.1.224.66:39661)
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:34:56.814623 2001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:35463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:34:56.833768 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:38431
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:45713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:56.951411 2555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:56.951702 2555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:56.951749 2555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:56.955849 2555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:56.955928 2555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:56.955991 2555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:34:56.961544 2555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:45713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:38431
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:34:56.962611 2555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:56.963621 2555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:56.974004 2566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:56.974004 2563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:56.973997 2562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:56.974227 2555 server_base.cc:1034] running on GCE node
I20260809 03:34:56.974809 2555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:56.975442 2555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:56.976567 2555 hybrid_clock.cc:648] HybridClock initialized: now 1786246496976555 us; error 27 us; skew 500 ppm
Aug 09 03:34:56 dist-test-slave-jq4t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:56.982594 2555 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:34:56.983632 2555 webserver.cc:460] Webserver started at http://127.1.224.67:42091/ using document root <none> and password file <none>
I20260809 03:34:56.984257 2555 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:56.984355 2555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:56.984673 2555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:34:56.986840 2555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "2d909ffbc62b49d4a2373c319238e691"
format_stamp: "Formatted at 2026-08-09 03:34:56 on dist-test-slave-jq4t"
server_key: "4bfb5fdffc12fd5a14e9947d46ce2d28"
server_key_iv: "c5c2491a11830321d43c365b631ecac0"
server_key_version: "encryptionkey@0"
I20260809 03:34:56.987445 2555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "2d909ffbc62b49d4a2373c319238e691"
format_stamp: "Formatted at 2026-08-09 03:34:56 on dist-test-slave-jq4t"
server_key: "4bfb5fdffc12fd5a14e9947d46ce2d28"
server_key_iv: "c5c2491a11830321d43c365b631ecac0"
server_key_version: "encryptionkey@0"
I20260809 03:34:56.991434 2555 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:34:56.994113 2580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:56.995464 2555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 03:34:56.995613 2555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "2d909ffbc62b49d4a2373c319238e691"
format_stamp: "Formatted at 2026-08-09 03:34:56 on dist-test-slave-jq4t"
server_key: "4bfb5fdffc12fd5a14e9947d46ce2d28"
server_key_iv: "c5c2491a11830321d43c365b631ecac0"
server_key_version: "encryptionkey@0"
I20260809 03:34:56.995735 2555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:57.007362 2555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:57.010648 2555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:57.010890 2555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:57.011543 2555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:57.012444 2555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:57.012544 2555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:57.012605 2555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:57.012622 2555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:57.021780 2555 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:34765
I20260809 03:34:57.021806 2771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:34765 every 8 connection(s)
I20260809 03:34:57.022780 2555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:34:57.031611 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 2555
I20260809 03:34:57.031764 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:34:57.032078 1921 external_mini_cluster.cc:1428] Setting key 61d175f5d638d7703ec3be576ce40702
Aug 09 03:34:57 dist-test-slave-jq4t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246496, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:34:57.037441 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.024849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:38337 (local address 127.1.224.126:38431)
0809 03:34:57.025014 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:57.025018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:57.025892 (+ 874us) server_negotiation.cc:407] Connection header received
0809 03:34:57.027262 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:57.027266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:57.027327 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:57.027408 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:57.028615 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.029182 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.030354 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.030589 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.030705 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.033854 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:57.033859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:57.033864 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:57.033895 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:57.035555 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:57.036038 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:57.036041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:57.036042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:57.036117 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:57.036422 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:57.036425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:57.036426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:57.036664 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:57.036746 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:57.037219 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 03:34:57.037318 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 03:34:57.038256 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.025280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:38431 (local address 127.1.224.67:38337)
0809 03:34:57.025746 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:57.025789 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:57.027050 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:57.027583 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:57.027589 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:57.027940 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:57.028424 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:57.028437 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.029315 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.029318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:57.030217 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:57.030227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.030746 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.030750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:57.030940 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.031619 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:57.031636 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:57.033630 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:57.035664 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:57.035669 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:57.035682 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:57.035953 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:57.036196 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:57.036198 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:57.036200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:57.036336 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:57.036756 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:57.036761 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:57.037014 (+ 253us) client_negotiation.cc:769] Sending connection context
0809 03:34:57.037173 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 03:34:57.037428 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":125,"threads_started":1}
I20260809 03:34:57.039331 2772 heartbeater.cc:347] Connected to a master server at 127.1.224.126:38431
I20260809 03:34:57.039587 2772 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:57.040087 2772 heartbeater.cc:511] Master 127.1.224.126:38431 requested a full tablet report, sending...
I20260809 03:34:57.041165 2001 ts_manager.cc:194] Registered new tserver with Master: 2d909ffbc62b49d4a2373c319238e691 (127.1.224.67:34765)
I20260809 03:34:57.041749 2001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:38337
I20260809 03:34:57.047168 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:34:57 dist-test-slave-jq4t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246497, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:34:57 dist-test-slave-jq4t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246497, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.224.65@KRBTEST.COM
I20260809 03:34:57.075680 2785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.065518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55750 (local address 127.1.224.65:42281)
0809 03:34:57.065830 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:57.065839 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:57.065872 (+ 33us) server_negotiation.cc:407] Connection header received
0809 03:34:57.065938 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:57.065947 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:57.066156 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:57.066321 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:57.067104 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.067880 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.068820 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.069217 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.069345 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.071906 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:57.071917 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:57.071933 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:57.071959 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:57.073723 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:57.074206 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:57.074212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:57.074214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:57.074302 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:57.074599 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:57.074604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:57.074605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:57.074893 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:57.075142 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:57.075323 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 03:34:57.075507 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":72,"threads_started":1}
W20260809 03:34:57.076915 2178 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55750
Aug 09 03:34:57 dist-test-slave-jq4t krb5kdc[1930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:34:57 dist-test-slave-jq4t krb5kdc[1930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.65@KRBTEST.COM
I20260809 03:34:57.106029 2785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:57.096763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55758 (local address 127.1.224.65:42281)
0809 03:34:57.096945 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:57.096951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:57.097183 (+ 232us) server_negotiation.cc:407] Connection header received
0809 03:34:57.097382 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:57.097385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:57.097432 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:57.097532 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:57.098287 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.098827 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.099763 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:57.100001 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:57.100120 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:57.102915 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:57.102919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:57.102921 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:57.102948 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:57.104363 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:57.104829 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:57.104832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:57.104833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:57.104948 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:57.105263 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:57.105265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:57.105266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:57.105443 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:57.105576 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:57.105787 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 03:34:57.105893 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:34:57.107381 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 2096
I20260809 03:34:57.122807 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 2341
I20260809 03:34:57.142081 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 2555
I20260809 03:34:57.158452 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 1951
2026-08-09T03:34:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2690 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:34:57 dist-test-slave-jq4t krb5kdc[2796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:34:57 dist-test-slave-jq4t krb5kdc[2796](info): set up 2 sockets
Aug 09 03:34:57 dist-test-slave-jq4t krb5kdc[2796](info): commencing operation
krb5kdc: starting...
W20260809 03:34:58.507925 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 03:34:58.626662 1921 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.119s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 03:34:58.795872 1921 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.169s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 03:34:58.961192 1921 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.165s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:34:58 dist-test-slave-jq4t krb5kdc[2796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:34:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:34:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:34:59.023978 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:40993
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:37921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:40993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:59.145270 2818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:59.145630 2818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:59.145737 2818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:59.149909 2818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:34:59.150008 2818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:59.150048 2818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:59.150105 2818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:34:59.150162 2818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:34:59.155781 2818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:37921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:40993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:40993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:34:59.157068 2818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:59.158169 2818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:59.167287 2832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:59.167330 2826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:59.167292 2825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:59.167623 2818 server_base.cc:1034] running on GCE node
I20260809 03:34:59.168299 2818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:59.169523 2818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:59.170717 2818 hybrid_clock.cc:648] HybridClock initialized: now 1786246499170700 us; error 31 us; skew 500 ppm
Aug 09 03:34:59 dist-test-slave-jq4t krb5kdc[2796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:59.173667 2818 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:34:59.174582 2818 webserver.cc:460] Webserver started at http://127.1.224.126:40349/ using document root <none> and password file <none>
I20260809 03:34:59.175169 2818 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:59.175264 2818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:59.175510 2818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:34:59.177690 2818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "3ca46d416c224b959f4f62b7ffeab9ff"
format_stamp: "Formatted at 2026-08-09 03:34:59 on dist-test-slave-jq4t"
server_key: "5a23a15d80440dfc41a97b24931b916b"
server_key_iv: "3e479b45402941e1f7a313176645f443"
server_key_version: "encryptionkey@0"
I20260809 03:34:59.178310 2818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "3ca46d416c224b959f4f62b7ffeab9ff"
format_stamp: "Formatted at 2026-08-09 03:34:59 on dist-test-slave-jq4t"
server_key: "5a23a15d80440dfc41a97b24931b916b"
server_key_iv: "3e479b45402941e1f7a313176645f443"
server_key_version: "encryptionkey@0"
I20260809 03:34:59.182322 2818 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 03:34:59.184856 2842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:59.186151 2818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:34:59.186321 2818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "3ca46d416c224b959f4f62b7ffeab9ff"
format_stamp: "Formatted at 2026-08-09 03:34:59 on dist-test-slave-jq4t"
server_key: "5a23a15d80440dfc41a97b24931b916b"
server_key_iv: "3e479b45402941e1f7a313176645f443"
server_key_version: "encryptionkey@0"
I20260809 03:34:59.186448 2818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:59.206641 2818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:59.209982 2818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:59.210207 2818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:59.217998 2818 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:40993
I20260809 03:34:59.218029 2932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:40993 every 8 connection(s)
I20260809 03:34:59.219022 2818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:34:59.221433 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 2818
I20260809 03:34:59.221534 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:34:59.221814 1921 external_mini_cluster.cc:1428] Setting key 70098b77aa6e27d66b83510eb931bb41
I20260809 03:34:59.222093 2933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:34:59.229807 2933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff: Bootstrap starting.
Aug 09 03:34:59 dist-test-slave-jq4t krb5kdc[2796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:34:59.232404 2933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff: Neither blocks nor log segments found. Creating new log.
I20260809 03:34:59.233440 2933 log.cc:824] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff: Log is configured to *not* fsync() on all Append() calls
I20260809 03:34:59.235994 2933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff: No bootstrap required, opened a new log
I20260809 03:34:59.238361 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:59.223373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41116 (local address 127.1.224.126:40993)
0809 03:34:59.223833 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:59.223866 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:59.223912 (+ 46us) server_negotiation.cc:407] Connection header received
0809 03:34:59.225076 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:59.225103 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:59.225532 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:59.225910 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:59.226595 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.228091 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.229012 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.229698 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.229861 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:59.232744 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:59.232752 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:59.232773 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:59.232808 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:59.234790 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:59.235236 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:59.235243 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:59.235249 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:59.235396 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:59.235674 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:59.235677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:59.235679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:59.236240 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:59.236444 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:59.236766 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 03:34:59.237096 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":138,"threads_started":1}
I20260809 03:34:59.239984 2933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca46d416c224b959f4f62b7ffeab9ff" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40993 } }
I20260809 03:34:59.240273 2933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:34:59.240345 2933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ca46d416c224b959f4f62b7ffeab9ff, State: Initialized, Role: FOLLOWER
I20260809 03:34:59.240875 2933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca46d416c224b959f4f62b7ffeab9ff" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40993 } }
I20260809 03:34:59.241051 2933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:34:59.241108 2933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:34:59.241184 2933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:34:59.242147 2933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca46d416c224b959f4f62b7ffeab9ff" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40993 } }
I20260809 03:34:59.242457 2933 leader_election.cc:302] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ca46d416c224b959f4f62b7ffeab9ff; no voters:
I20260809 03:34:59.242748 2933 leader_election.cc:288] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:34:59.242887 2939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:34:59.243160 2939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [term 1 LEADER]: Becoming Leader. State: Replica: 3ca46d416c224b959f4f62b7ffeab9ff, State: Running, Role: LEADER
I20260809 03:34:59.243480 2939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca46d416c224b959f4f62b7ffeab9ff" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40993 } }
I20260809 03:34:59.244076 2933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:34:59.244678 2939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ca46d416c224b959f4f62b7ffeab9ff. Latest consensus state: current_term: 1 leader_uuid: "3ca46d416c224b959f4f62b7ffeab9ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca46d416c224b959f4f62b7ffeab9ff" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40993 } } }
I20260809 03:34:59.244818 2939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [sys.catalog]: This master's current role is: LEADER
I20260809 03:34:59.245229 2950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:34:59.245751 2947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ca46d416c224b959f4f62b7ffeab9ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca46d416c224b959f4f62b7ffeab9ff" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40993 } } }
I20260809 03:34:59.245862 2947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff [sys.catalog]: This master's current role is: LEADER
I20260809 03:34:59.248366 2950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:34:59.254742 2950 catalog_manager.cc:1347] Generated new cluster ID: dee68cab866647fd8d6a29da45a29f43
I20260809 03:34:59.254832 2950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:34:59.263717 2950 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:34:59.265059 2950 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:34:59.270081 2950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ca46d416c224b959f4f62b7ffeab9ff: Generated new TSK 0
I20260809 03:34:59.270891 2950 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:34:59.303224 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40993
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:37921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:59.426061 2972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:59.426398 2972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:59.426486 2972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:59.430593 2972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:59.430703 2972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:59.430802 2972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:34:59.436393 2972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:37921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40993
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:34:59.437721 2972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:59.438874 2972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:59.449198 2986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:59.449203 2990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:59.449242 2985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:59.449445 2972 server_base.cc:1034] running on GCE node
I20260809 03:34:59.450016 2972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:59.450668 2972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:59.451861 2972 hybrid_clock.cc:648] HybridClock initialized: now 1786246499451850 us; error 34 us; skew 500 ppm
Aug 09 03:34:59 dist-test-slave-jq4t krb5kdc[2796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:59.454680 2972 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:34:59.455616 2972 webserver.cc:460] Webserver started at http://127.1.224.65:36169/ using document root <none> and password file <none>
I20260809 03:34:59.456218 2972 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:59.456311 2972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:59.456622 2972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:34:59.459136 2972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "f730b6aef6034e07b8510e2d1944c2ce"
format_stamp: "Formatted at 2026-08-09 03:34:59 on dist-test-slave-jq4t"
server_key: "3d0c92041717a08c1ac15dced1454ece"
server_key_iv: "a43f4e16aaa83590684f1598158a48ec"
server_key_version: "encryptionkey@0"
I20260809 03:34:59.459729 2972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "f730b6aef6034e07b8510e2d1944c2ce"
format_stamp: "Formatted at 2026-08-09 03:34:59 on dist-test-slave-jq4t"
server_key: "3d0c92041717a08c1ac15dced1454ece"
server_key_iv: "a43f4e16aaa83590684f1598158a48ec"
server_key_version: "encryptionkey@0"
I20260809 03:34:59.463550 2972 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:34:59.466042 3003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:59.467188 2972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 03:34:59.467326 2972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "f730b6aef6034e07b8510e2d1944c2ce"
format_stamp: "Formatted at 2026-08-09 03:34:59 on dist-test-slave-jq4t"
server_key: "3d0c92041717a08c1ac15dced1454ece"
server_key_iv: "a43f4e16aaa83590684f1598158a48ec"
server_key_version: "encryptionkey@0"
I20260809 03:34:59.467438 2972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:59.479557 2972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:59.482779 2972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:59.483013 2972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:59.483650 2972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:59.484625 2972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:59.484694 2972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:59.484763 2972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:59.484813 2972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:59.494592 2972 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:34699
I20260809 03:34:59.494617 3202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:34699 every 8 connection(s)
I20260809 03:34:59.495576 2972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:34:59.501490 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 2972
I20260809 03:34:59.501607 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:34:59.501865 1921 external_mini_cluster.cc:1428] Setting key 1726b82e3d3d8aa630eb77e4fb6f64e4
Aug 09 03:34:59 dist-test-slave-jq4t krb5kdc[2796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:34:59.510355 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:59.497507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:39909 (local address 127.1.224.126:40993)
0809 03:34:59.497663 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:59.497667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:59.498525 (+ 858us) server_negotiation.cc:407] Connection header received
0809 03:34:59.499829 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:59.499833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:59.499880 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:59.499957 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:59.501250 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.501923 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.503292 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.503640 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.503761 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:59.506479 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:59.506482 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:59.506484 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:59.506509 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:59.507961 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:59.508685 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:59.508689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:59.508690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:59.508769 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:59.509150 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:59.509153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:59.509155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:59.509358 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:59.509466 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:59.510025 (+ 559us) server_negotiation.cc:299] Negotiation successful
0809 03:34:59.510171 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:34:59.511258 3207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:59.497879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:40993 (local address 127.1.224.65:39909)
0809 03:34:59.498351 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:59.498399 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:59.499631 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:59.500126 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:59.500132 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:59.500517 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:59.501063 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:59.501080 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.502076 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.502081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:59.503166 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:59.503180 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.503817 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.503821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:59.503988 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:59.504703 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:59.504720 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:59.506291 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:59.508099 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:59.508106 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:59.508123 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:59.508561 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:59.508867 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:59.508871 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:59.508873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:59.509042 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:59.509470 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:59.509479 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:59.509763 (+ 284us) client_negotiation.cc:769] Sending connection context
0809 03:34:59.509964 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 03:34:59.510331 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":118,"threads_started":1}
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:34:59.512668 3203 heartbeater.cc:347] Connected to a master server at 127.1.224.126:40993
I20260809 03:34:59.512948 3203 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:59.513512 3203 heartbeater.cc:511] Master 127.1.224.126:40993 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:34:59.515367 2877 ts_manager.cc:194] Registered new tserver with Master: f730b6aef6034e07b8510e2d1944c2ce (127.1.224.65:34699)
I20260809 03:34:59.516710 2877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:39909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:34:59.532302 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40993
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:37921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:59.653959 3212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:59.654227 3212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:59.654271 3212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:59.658308 3212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:59.658365 3212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:59.658424 3212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:34:59.663726 3212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:37921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40993
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:34:59.664836 3212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:59.665858 3212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:59.675859 3225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:59.675808 3227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:59.675912 3224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:59.676132 3212 server_base.cc:1034] running on GCE node
I20260809 03:34:59.676721 3212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:59.677348 3212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:59.678535 3212 hybrid_clock.cc:648] HybridClock initialized: now 1786246499678519 us; error 27 us; skew 500 ppm
Aug 09 03:34:59 dist-test-slave-jq4t krb5kdc[2796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:59.681989 3212 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:34:59.682947 3212 webserver.cc:460] Webserver started at http://127.1.224.66:44241/ using document root <none> and password file <none>
I20260809 03:34:59.683571 3212 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:59.683667 3212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:59.683920 3212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:34:59.686146 3212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "bce608d508d34c09944b431805cae1cf"
format_stamp: "Formatted at 2026-08-09 03:34:59 on dist-test-slave-jq4t"
server_key: "1384979bc1051445b2cb39697e64a2c3"
server_key_iv: "fdfe4d7b800b9e9ef7d8a4004251fefa"
server_key_version: "encryptionkey@0"
I20260809 03:34:59.686746 3212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "bce608d508d34c09944b431805cae1cf"
format_stamp: "Formatted at 2026-08-09 03:34:59 on dist-test-slave-jq4t"
server_key: "1384979bc1051445b2cb39697e64a2c3"
server_key_iv: "fdfe4d7b800b9e9ef7d8a4004251fefa"
server_key_version: "encryptionkey@0"
I20260809 03:34:59.690990 3212 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:34:59.693729 3244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:59.695127 3212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:34:59.695290 3212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "bce608d508d34c09944b431805cae1cf"
format_stamp: "Formatted at 2026-08-09 03:34:59 on dist-test-slave-jq4t"
server_key: "1384979bc1051445b2cb39697e64a2c3"
server_key_iv: "fdfe4d7b800b9e9ef7d8a4004251fefa"
server_key_version: "encryptionkey@0"
I20260809 03:34:59.695431 3212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:59.731513 3212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:59.734952 3212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:59.735188 3212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:59.735860 3212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:59.736850 3212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:59.736927 3212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:59.737041 3212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:59.737097 3212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:59.747025 3212 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:41257
I20260809 03:34:59.747078 3433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:41257 every 8 connection(s)
I20260809 03:34:59.748062 3212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:34:59.751003 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 3212
I20260809 03:34:59.751138 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:34:59.751483 1921 external_mini_cluster.cc:1428] Setting key 39aebdb1eb2f3e6f98e11343544e88e9
Aug 09 03:34:59 dist-test-slave-jq4t krb5kdc[2796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:34:59.764252 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:59.750191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:50009 (local address 127.1.224.126:40993)
0809 03:34:59.750349 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:59.750355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:59.751345 (+ 990us) server_negotiation.cc:407] Connection header received
0809 03:34:59.753064 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:59.753067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:59.753125 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:59.753205 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:59.754448 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.755012 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.756223 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.756468 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.756614 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:59.759772 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:59.759780 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:59.759785 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:59.759828 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:59.761707 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:59.762333 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:59.762339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:59.762341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:59.762449 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:59.762820 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:59.762827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:59.762829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:59.763137 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:34:59.763266 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 03:34:59.763864 (+ 598us) server_negotiation.cc:299] Negotiation successful
0809 03:34:59.764039 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:34:59.765224 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:59.750659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:40993 (local address 127.1.224.66:50009)
0809 03:34:59.751169 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:59.751224 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:59.752761 (+ 1537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:59.753390 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:59.753398 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:59.753779 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:59.754297 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:59.754310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.755159 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.755162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:59.756084 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:59.756094 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.756673 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.756677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:59.756922 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:59.757614 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:59.757642 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:59.759560 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:59.761856 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:59.761863 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:59.761885 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:59.762213 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:59.762558 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:59.762561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:59.762562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:59.762709 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:59.763271 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:34:59.763277 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:34:59.763574 (+ 297us) client_negotiation.cc:769] Sending connection context
0809 03:34:59.763754 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 03:34:59.764135 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":143,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
I20260809 03:34:59.766645 3434 heartbeater.cc:347] Connected to a master server at 127.1.224.126:40993
I20260809 03:34:59.766948 3434 heartbeater.cc:464] Registering TS with master...
I20260809 03:34:59.767638 3434 heartbeater.cc:511] Master 127.1.224.126:40993 requested a full tablet report, sending...
I20260809 03:34:59.769006 2877 ts_manager.cc:194] Registered new tserver with Master: bce608d508d34c09944b431805cae1cf (127.1.224.66:41257)
I20260809 03:34:59.769634 2877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:50009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:34:59.784031 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40993
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:37921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:34:59.903743 3443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:34:59.904074 3443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:34:59.904150 3443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:34:59.908351 3443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:34:59.908448 3443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:34:59.908574 3443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:34:59.914093 3443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:37921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40993
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:34:59.915333 3443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:34:59.916493 3443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:34:59.926764 3452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:34:59.926771 3450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:59.926942 3443 server_base.cc:1034] running on GCE node
W20260809 03:34:59.926771 3460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:34:59.927518 3443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:34:59.928231 3443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:34:59.929425 3443 hybrid_clock.cc:648] HybridClock initialized: now 1786246499929392 us; error 44 us; skew 500 ppm
Aug 09 03:34:59 dist-test-slave-jq4t krb5kdc[2796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:34:59.932349 3443 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:34:59.933311 3443 webserver.cc:460] Webserver started at http://127.1.224.67:33955/ using document root <none> and password file <none>
I20260809 03:34:59.933929 3443 fs_manager.cc:359] Metadata directory not provided
I20260809 03:34:59.934021 3443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:34:59.934275 3443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:34:59.936399 3443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "99581293ddfc4932be7e89e9456bc69f"
format_stamp: "Formatted at 2026-08-09 03:34:59 on dist-test-slave-jq4t"
server_key: "f3c4bb3dfb00be3ba27f29a41566b4dc"
server_key_iv: "ee9600bbd9134771a5637f3d0fd91912"
server_key_version: "encryptionkey@0"
I20260809 03:34:59.937083 3443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "99581293ddfc4932be7e89e9456bc69f"
format_stamp: "Formatted at 2026-08-09 03:34:59 on dist-test-slave-jq4t"
server_key: "f3c4bb3dfb00be3ba27f29a41566b4dc"
server_key_iv: "ee9600bbd9134771a5637f3d0fd91912"
server_key_version: "encryptionkey@0"
I20260809 03:34:59.940893 3443 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:34:59.943384 3471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:59.944722 3443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:34:59.944900 3443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "99581293ddfc4932be7e89e9456bc69f"
format_stamp: "Formatted at 2026-08-09 03:34:59 on dist-test-slave-jq4t"
server_key: "f3c4bb3dfb00be3ba27f29a41566b4dc"
server_key_iv: "ee9600bbd9134771a5637f3d0fd91912"
server_key_version: "encryptionkey@0"
I20260809 03:34:59.945017 3443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:34:59.970551 3443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:34:59.973987 3443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:34:59.974231 3443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:34:59.974844 3443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:34:59.975664 3443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:34:59.975711 3443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:59.975752 3443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:34:59.975775 3443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:34:59.985172 3443 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:37019
I20260809 03:34:59.985191 3657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:37019 every 8 connection(s)
I20260809 03:34:59.986214 3443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:34:59.992197 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 3443
I20260809 03:34:59.992312 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:34:59.992676 1921 external_mini_cluster.cc:1428] Setting key d9ee9117d12a94118855038e3f4c9ef6
Aug 09 03:34:59 dist-test-slave-jq4t krb5kdc[2796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:00.001493 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:59.988424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:48625 (local address 127.1.224.126:40993)
0809 03:34:59.988679 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 03:34:59.988684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:34:59.989618 (+ 934us) server_negotiation.cc:407] Connection header received
0809 03:34:59.991201 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:34:59.991205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:34:59.991271 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:34:59.991384 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:34:59.992912 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.993588 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.995040 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.995266 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.995349 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:59.997948 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:34:59.997953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:34:59.997956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:34:59.997983 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:34:59.999285 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:34:59.999784 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:34:59.999787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:34:59.999789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:34:59.999872 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.000250 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.000253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.000254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.000450 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.000621 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.001244 (+ 623us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.001362 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260809 03:35:00.002391 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:34:59.988870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:40993 (local address 127.1.224.67:48625)
0809 03:34:59.989424 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 03:34:59.989474 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 03:34:59.990955 (+ 1481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:34:59.991590 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:34:59.991598 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:34:59.992001 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:34:59.992656 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:59.992674 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.993932 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.993935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:59.994915 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:34:59.994927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:34:59.995422 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:34:59.995425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:34:59.995589 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:34:59.996192 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:34:59.996210 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:34:59.997783 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:34:59.999394 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:59.999399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:59.999412 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:34:59.999679 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:34:59.999984 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:34:59.999986 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:34:59.999988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:00.000123 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:00.000620 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:00.000625 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:00.000931 (+ 306us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.001107 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.001379 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":124,"threads_started":1}
I20260809 03:35:00.003597 3659 heartbeater.cc:347] Connected to a master server at 127.1.224.126:40993
I20260809 03:35:00.003859 3659 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:00.004408 3659 heartbeater.cc:511] Master 127.1.224.126:40993 requested a full tablet report, sending...
I20260809 03:35:00.005894 2877 ts_manager.cc:194] Registered new tserver with Master: 99581293ddfc4932be7e89e9456bc69f (127.1.224.67:37019)
I20260809 03:35:00.006757 2877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:48625
I20260809 03:35:00.007011 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:35:00.018958 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.011447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.1.224.126:40993)
0809 03:35:00.011652 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.011657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.011761 (+ 104us) server_negotiation.cc:407] Connection header received
0809 03:35:00.011864 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.011867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.011909 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.011975 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.012850 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.013379 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.014192 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.014409 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.014497 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.015906 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.015909 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.015911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.015936 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.017339 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.017738 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.017741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.017742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.017821 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.018075 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.018077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.018078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.018303 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.018417 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.018627 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.018729 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260809 03:35:00.025770 2877 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:00.029332 2877 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:00.046505 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.039907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:41257 (local address 127.0.0.1:59088)
0809 03:35:00.040408 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.040466 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.040634 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.041452 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.041459 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.041484 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:00.041649 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.041656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.043095 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.043102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.044397 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.044408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.045619 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.045623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.045739 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.045784 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.046184 (+ 400us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.046324 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":26,"thread_start_us":132,"threads_started":1}
I20260809 03:35:00.046706 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.041374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.1.224.65:34699)
0809 03:35:00.041773 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.041780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.041800 (+ 20us) server_negotiation.cc:407] Connection header received
0809 03:35:00.041880 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.041887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.042101 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.042295 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:00.042544 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.043642 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.045267 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.046064 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.046198 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.046339 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.046446 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.046545 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":112,"threads_started":1}
I20260809 03:35:00.046787 3686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.040116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59088 (local address 127.1.224.66:41257)
0809 03:35:00.040960 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.040975 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.041002 (+ 27us) server_negotiation.cc:407] Connection header received
0809 03:35:00.041083 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.041087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.041249 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.041806 (+ 557us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:00.041995 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.042961 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.044597 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.045473 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.045590 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.045814 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.046526 (+ 712us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.046637 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":130,"threads_started":1}
I20260809 03:35:00.046905 3689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.040262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:37019 (local address 127.0.0.1:57218)
0809 03:35:00.040835 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.040849 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.040930 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.042056 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.042059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.042075 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:00.042201 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.042210 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.043755 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.043758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.045059 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.045069 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.046185 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.046188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.046596 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.046619 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.046692 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.046771 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":368,"thread_start_us":118,"threads_started":1}
I20260809 03:35:00.047029 3690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.040746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:34699 (local address 127.0.0.1:53352)
0809 03:35:00.041052 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.041067 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.041158 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.042297 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.042301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.042322 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:00.042403 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.042408 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.043787 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.043791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.045114 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.045142 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.046201 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.046204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.046309 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.046332 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.046414 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.046485 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":22,"thread_start_us":102,"threads_started":1}
I20260809 03:35:00.047637 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.041122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.1.224.67:37019)
0809 03:35:00.041516 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.041528 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.041577 (+ 49us) server_negotiation.cc:407] Connection header received
0809 03:35:00.041653 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.041658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.041849 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.042062 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:00.042337 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.043591 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.045187 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.046013 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.046284 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.046431 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.047054 (+ 623us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.047156 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":158,"threads_started":1}
I20260809 03:35:00.050668 3316 tablet_service.cc:1467] Processing CreateTablet for tablet 8d85ae5fb8b340668f7d6f80b2643e13 (DEFAULT_TABLE table=test-table [id=0a32fa3b9d7d4374bf52a09cac8ecad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:00.051919 3316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d85ae5fb8b340668f7d6f80b2643e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:00.052692 3552 tablet_service.cc:1467] Processing CreateTablet for tablet 8d85ae5fb8b340668f7d6f80b2643e13 (DEFAULT_TABLE table=test-table [id=0a32fa3b9d7d4374bf52a09cac8ecad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:00.053205 3090 tablet_service.cc:1467] Processing CreateTablet for tablet 8d85ae5fb8b340668f7d6f80b2643e13 (DEFAULT_TABLE table=test-table [id=0a32fa3b9d7d4374bf52a09cac8ecad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:00.053853 3552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d85ae5fb8b340668f7d6f80b2643e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:00.054293 3090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d85ae5fb8b340668f7d6f80b2643e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:00.059247 3697 tablet_bootstrap.cc:492] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf: Bootstrap starting.
I20260809 03:35:00.061257 3699 tablet_bootstrap.cc:492] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce: Bootstrap starting.
I20260809 03:35:00.062239 3701 tablet_bootstrap.cc:492] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f: Bootstrap starting.
I20260809 03:35:00.062672 3697 tablet_bootstrap.cc:654] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:00.063612 3697 log.cc:824] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:00.063786 3699 tablet_bootstrap.cc:654] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:00.064604 3699 log.cc:824] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:00.064925 3701 tablet_bootstrap.cc:654] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:00.065594 3701 log.cc:824] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:00.065878 3697 tablet_bootstrap.cc:492] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf: No bootstrap required, opened a new log
I20260809 03:35:00.066118 3697 ts_tablet_manager.cc:1397] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 03:35:00.067368 3701 tablet_bootstrap.cc:492] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f: No bootstrap required, opened a new log
I20260809 03:35:00.067579 3701 ts_tablet_manager.cc:1397] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:35:00.069447 3697 raft_consensus.cc:348] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bce608d508d34c09944b431805cae1cf" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 41257 } } peers { permanent_uuid: "f730b6aef6034e07b8510e2d1944c2ce" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34699 } } peers { permanent_uuid: "99581293ddfc4932be7e89e9456bc69f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37019 } }
I20260809 03:35:00.069754 3697 raft_consensus.cc:374] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:00.069818 3697 raft_consensus.cc:729] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bce608d508d34c09944b431805cae1cf, State: Initialized, Role: FOLLOWER
I20260809 03:35:00.070341 3699 tablet_bootstrap.cc:492] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce: No bootstrap required, opened a new log
I20260809 03:35:00.070412 3697 consensus_queue.cc:260] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bce608d508d34c09944b431805cae1cf" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 41257 } } peers { permanent_uuid: "f730b6aef6034e07b8510e2d1944c2ce" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34699 } } peers { permanent_uuid: "99581293ddfc4932be7e89e9456bc69f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37019 } }
I20260809 03:35:00.070538 3699 ts_tablet_manager.cc:1397] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260809 03:35:00.071074 3701 raft_consensus.cc:348] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bce608d508d34c09944b431805cae1cf" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 41257 } } peers { permanent_uuid: "f730b6aef6034e07b8510e2d1944c2ce" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34699 } } peers { permanent_uuid: "99581293ddfc4932be7e89e9456bc69f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37019 } }
I20260809 03:35:00.071343 3701 raft_consensus.cc:374] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:00.071455 3701 raft_consensus.cc:729] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99581293ddfc4932be7e89e9456bc69f, State: Initialized, Role: FOLLOWER
I20260809 03:35:00.071568 3697 ts_tablet_manager.cc:1428] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 03:35:00.071883 3701 consensus_queue.cc:260] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bce608d508d34c09944b431805cae1cf" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 41257 } } peers { permanent_uuid: "f730b6aef6034e07b8510e2d1944c2ce" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34699 } } peers { permanent_uuid: "99581293ddfc4932be7e89e9456bc69f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37019 } }
I20260809 03:35:00.072109 3434 heartbeater.cc:503] Master 127.1.224.126:40993 was elected leader, sending a full tablet report...
I20260809 03:35:00.072933 3701 ts_tablet_manager.cc:1428] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 03:35:00.073002 3659 heartbeater.cc:503] Master 127.1.224.126:40993 was elected leader, sending a full tablet report...
I20260809 03:35:00.075428 3699 raft_consensus.cc:348] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bce608d508d34c09944b431805cae1cf" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 41257 } } peers { permanent_uuid: "f730b6aef6034e07b8510e2d1944c2ce" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34699 } } peers { permanent_uuid: "99581293ddfc4932be7e89e9456bc69f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37019 } }
I20260809 03:35:00.075721 3699 raft_consensus.cc:374] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:00.075789 3699 raft_consensus.cc:729] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f730b6aef6034e07b8510e2d1944c2ce, State: Initialized, Role: FOLLOWER
I20260809 03:35:00.076216 3699 consensus_queue.cc:260] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bce608d508d34c09944b431805cae1cf" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 41257 } } peers { permanent_uuid: "f730b6aef6034e07b8510e2d1944c2ce" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34699 } } peers { permanent_uuid: "99581293ddfc4932be7e89e9456bc69f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37019 } }
I20260809 03:35:00.077241 3699 ts_tablet_manager.cc:1428] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260809 03:35:00.077237 3203 heartbeater.cc:503] Master 127.1.224.126:40993 was elected leader, sending a full tablet report...
I20260809 03:35:00.203471 3708 raft_consensus.cc:482] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:00.203707 3708 raft_consensus.cc:504] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bce608d508d34c09944b431805cae1cf" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 41257 } } peers { permanent_uuid: "f730b6aef6034e07b8510e2d1944c2ce" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34699 } } peers { permanent_uuid: "99581293ddfc4932be7e89e9456bc69f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37019 } }
I20260809 03:35:00.205042 3708 leader_election.cc:288] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bce608d508d34c09944b431805cae1cf (127.1.224.66:41257), f730b6aef6034e07b8510e2d1944c2ce (127.1.224.65:34699)
I20260809 03:35:00.210372 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.205871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:54655 (local address 127.1.224.65:34699)
0809 03:35:00.206035 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.206040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.206178 (+ 138us) server_negotiation.cc:407] Connection header received
0809 03:35:00.206296 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.206298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.206341 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.206428 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:00.206680 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.207391 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.208993 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.209621 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.209748 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.209807 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.210156 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.210231 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:35:00.210372 3686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.205494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:59331 (local address 127.1.224.66:41257)
0809 03:35:00.205662 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.205667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.205688 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:35:00.206004 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.206007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.206056 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.206128 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:00.206450 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.207250 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.208990 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.209618 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.209748 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.209806 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.210156 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.210231 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 03:35:00.210433 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.205345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:41257 (local address 127.1.224.67:59331)
0809 03:35:00.205515 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.205535 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.205681 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.206150 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.206153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.206175 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:00.206329 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.206335 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.207360 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.207362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.208835 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.208848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.209749 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.209753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.209956 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.210038 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.210122 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.210244 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":158}
I20260809 03:35:00.210644 3715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.205663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:34699 (local address 127.1.224.67:54655)
0809 03:35:00.206073 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:00.206087 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:00.206189 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:00.206448 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:00.206451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:00.206477 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:00.206572 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.206576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.207508 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.207511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.208835 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:00.208848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.209749 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.209753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:00.209960 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.210042 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 03:35:00.210122 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 03:35:00.210243 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":158,"spinlock_wait_cycles":6912,"thread_start_us":160,"threads_started":1}
I20260809 03:35:00.211347 3124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d85ae5fb8b340668f7d6f80b2643e13" candidate_uuid: "99581293ddfc4932be7e89e9456bc69f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f730b6aef6034e07b8510e2d1944c2ce" is_pre_election: true
I20260809 03:35:00.211807 3124 raft_consensus.cc:2454] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99581293ddfc4932be7e89e9456bc69f in term 0.
I20260809 03:35:00.212383 3475 leader_election.cc:302] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99581293ddfc4932be7e89e9456bc69f, f730b6aef6034e07b8510e2d1944c2ce; no voters:
I20260809 03:35:00.212723 3708 raft_consensus.cc:2781] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:00.212805 3708 raft_consensus.cc:482] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:00.212836 3708 raft_consensus.cc:3037] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:00.213626 3354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d85ae5fb8b340668f7d6f80b2643e13" candidate_uuid: "99581293ddfc4932be7e89e9456bc69f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bce608d508d34c09944b431805cae1cf" is_pre_election: true
I20260809 03:35:00.213970 3354 raft_consensus.cc:2454] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99581293ddfc4932be7e89e9456bc69f in term 0.
I20260809 03:35:00.214172 3708 raft_consensus.cc:504] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bce608d508d34c09944b431805cae1cf" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 41257 } } peers { permanent_uuid: "f730b6aef6034e07b8510e2d1944c2ce" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34699 } } peers { permanent_uuid: "99581293ddfc4932be7e89e9456bc69f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37019 } }
I20260809 03:35:00.214529 3708 leader_election.cc:288] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [CANDIDATE]: Term 1 election: Requested vote from peers bce608d508d34c09944b431805cae1cf (127.1.224.66:41257), f730b6aef6034e07b8510e2d1944c2ce (127.1.224.65:34699)
I20260809 03:35:00.215083 3354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d85ae5fb8b340668f7d6f80b2643e13" candidate_uuid: "99581293ddfc4932be7e89e9456bc69f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bce608d508d34c09944b431805cae1cf"
I20260809 03:35:00.215183 3124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d85ae5fb8b340668f7d6f80b2643e13" candidate_uuid: "99581293ddfc4932be7e89e9456bc69f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f730b6aef6034e07b8510e2d1944c2ce"
I20260809 03:35:00.215193 3354 raft_consensus.cc:3037] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:00.215327 3124 raft_consensus.cc:3037] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:00.216349 3354 raft_consensus.cc:2454] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99581293ddfc4932be7e89e9456bc69f in term 1.
I20260809 03:35:00.216470 3124 raft_consensus.cc:2454] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99581293ddfc4932be7e89e9456bc69f in term 1.
I20260809 03:35:00.216771 3475 leader_election.cc:302] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99581293ddfc4932be7e89e9456bc69f, bce608d508d34c09944b431805cae1cf; no voters:
I20260809 03:35:00.216953 3708 raft_consensus.cc:2781] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:00.217188 3708 raft_consensus.cc:686] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [term 1 LEADER]: Becoming Leader. State: Replica: 99581293ddfc4932be7e89e9456bc69f, State: Running, Role: LEADER
I20260809 03:35:00.217854 3708 consensus_queue.cc:237] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bce608d508d34c09944b431805cae1cf" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 41257 } } peers { permanent_uuid: "f730b6aef6034e07b8510e2d1944c2ce" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34699 } } peers { permanent_uuid: "99581293ddfc4932be7e89e9456bc69f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37019 } }
I20260809 03:35:00.221714 2877 catalog_manager.cc:5495] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f reported cstate change: term changed from 0 to 1, leader changed from <none> to 99581293ddfc4932be7e89e9456bc69f (127.1.224.67). New cstate: current_term: 1 leader_uuid: "99581293ddfc4932be7e89e9456bc69f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bce608d508d34c09944b431805cae1cf" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 41257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f730b6aef6034e07b8510e2d1944c2ce" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99581293ddfc4932be7e89e9456bc69f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37019 } health_report { overall_health: HEALTHY } } }
Aug 09 03:35:00 dist-test-slave-jq4t krb5kdc[2796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.65@KRBTEST.COM
W20260809 03:35:00.237394 3660 tablet.cc:2369] T 8d85ae5fb8b340668f7d6f80b2643e13 P 99581293ddfc4932be7e89e9456bc69f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:00.239933 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.231015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53366 (local address 127.1.224.65:34699)
0809 03:35:00.231171 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.231176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.231342 (+ 166us) server_negotiation.cc:407] Connection header received
0809 03:35:00.231463 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.231465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.231507 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.231576 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.232373 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.232947 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.233759 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.233968 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.234053 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.236540 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.236554 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.236569 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.236595 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.238296 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.238714 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.238719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.238720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.238805 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.239040 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.239042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.239043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.239282 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.239439 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.239608 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.239731 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 03:35:00.246882 3205 tablet.cc:2369] T 8d85ae5fb8b340668f7d6f80b2643e13 P f730b6aef6034e07b8510e2d1944c2ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:35:00.249501 3435 tablet.cc:2369] T 8d85ae5fb8b340668f7d6f80b2643e13 P bce608d508d34c09944b431805cae1cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 03:35:00 dist-test-slave-jq4t krb5kdc[2796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:35:00 dist-test-slave-jq4t krb5kdc[2796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.224.65@KRBTEST.COM
I20260809 03:35:00.268407 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.260252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53380 (local address 127.1.224.65:34699)
0809 03:35:00.260397 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.260401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.260604 (+ 203us) server_negotiation.cc:407] Connection header received
0809 03:35:00.260729 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.260731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.260774 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.260850 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.261526 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.262039 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.262834 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.263047 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.263121 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.265512 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.265516 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.265517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.265545 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.267021 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.267377 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.267379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.267381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.267453 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.267676 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.267678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.267679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.267833 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.267940 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.268188 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.268283 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 03:35:00 dist-test-slave-jq4t krb5kdc[2796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:35:00 dist-test-slave-jq4t krb5kdc[2796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.65@KRBTEST.COM
I20260809 03:35:00.295466 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:00.286669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.1.224.65:34699)
0809 03:35:00.286834 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:00.286838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:00.286853 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:35:00.286980 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:00.286983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:00.287026 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:00.287151 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:00.288122 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.288779 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.289650 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:00.289874 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:00.289998 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:00.292437 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:00.292440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:00.292442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:00.292468 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:00.293775 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.294210 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.294219 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.294220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.294300 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:00.294598 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:00.294602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:00.294604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:00.294787 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:00.294924 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:00.295171 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 03:35:00.295298 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 03:35:00.297209 3053 tablet_service.cc:2908] Rejecting scan request for tablet 8d85ae5fb8b340668f7d6f80b2643e13: Uninitialized: clean time has not yet been initialized
I20260809 03:35:00.299472 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 2972
I20260809 03:35:00.313478 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 3212
I20260809 03:35:00.326473 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 3443
I20260809 03:35:00.349049 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 2818
2026-08-09T03:35:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3193 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:00 dist-test-slave-jq4t krb5kdc[3743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:00 dist-test-slave-jq4t krb5kdc[3743](info): set up 2 sockets
Aug 09 03:35:00 dist-test-slave-jq4t krb5kdc[3743](info): commencing operation
krb5kdc: starting...
W20260809 03:35:01.626617 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:35:01 dist-test-slave-jq4t krb5kdc[3743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:35:01.760236 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:39521
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:39521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:01.880517 3764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:01.880828 3764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:01.880914 3764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:01.885033 3764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:01.885121 3764 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 03:35:01.885150 3764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:01.885197 3764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:01.885250 3764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:01.885288 3764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:01.890650 3764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:39521
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:39521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:01.891839 3764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:01.893015 3764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:01.902168 3772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:01.902267 3764 server_base.cc:1034] running on GCE node
W20260809 03:35:01.902181 3773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:01.902180 3775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:01.903172 3764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:01.904358 3764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:01.905557 3764 hybrid_clock.cc:648] HybridClock initialized: now 1786246501905529 us; error 46 us; skew 500 ppm
Aug 09 03:35:01 dist-test-slave-jq4t krb5kdc[3743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246501, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:01.912089 3764 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:35:01.913172 3764 webserver.cc:460] Webserver started at http://127.1.224.126:39327/ using document root <none> and password file <none>
I20260809 03:35:01.913795 3764 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:01.913892 3764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:01.914152 3764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:01.916352 3764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "47bf3b076830404b9745cd50a6a440eb"
format_stamp: "Formatted at 2026-08-09 03:35:01 on dist-test-slave-jq4t"
server_key: "ad2158afa20ef3dbbff2745e30ae1b54"
server_key_iv: "f350c0b72de2c5fa6003849f3485f58d"
server_key_version: "encryptionkey@0"
I20260809 03:35:01.917039 3764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "47bf3b076830404b9745cd50a6a440eb"
format_stamp: "Formatted at 2026-08-09 03:35:01 on dist-test-slave-jq4t"
server_key: "ad2158afa20ef3dbbff2745e30ae1b54"
server_key_iv: "f350c0b72de2c5fa6003849f3485f58d"
server_key_version: "encryptionkey@0"
I20260809 03:35:01.920967 3764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:35:01.923424 3787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:01.924705 3764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:01.924850 3764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "47bf3b076830404b9745cd50a6a440eb"
format_stamp: "Formatted at 2026-08-09 03:35:01 on dist-test-slave-jq4t"
server_key: "ad2158afa20ef3dbbff2745e30ae1b54"
server_key_iv: "f350c0b72de2c5fa6003849f3485f58d"
server_key_version: "encryptionkey@0"
I20260809 03:35:01.924978 3764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:01.955669 3764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:01.961221 3764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:01.961448 3764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:01.973930 3764 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:39521
I20260809 03:35:01.973978 3891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:39521 every 8 connection(s)
I20260809 03:35:01.975412 3764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:01.979377 3892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:01.979565 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 3764
I20260809 03:35:01.979705 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:01.980075 1921 external_mini_cluster.cc:1428] Setting key 870b72858824d9f195d85e741a84317e
I20260809 03:35:01.987636 3892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb: Bootstrap starting.
I20260809 03:35:01.990340 3892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:01.991183 3892 log.cc:824] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:35:01 dist-test-slave-jq4t krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:01.993521 3892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb: No bootstrap required, opened a new log
I20260809 03:35:01.996940 3892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf3b076830404b9745cd50a6a440eb" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39521 } }
I20260809 03:35:01.997213 3892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:01.997303 3892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47bf3b076830404b9745cd50a6a440eb, State: Initialized, Role: FOLLOWER
I20260809 03:35:01.997828 3892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf3b076830404b9745cd50a6a440eb" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39521 } }
I20260809 03:35:01.997991 3892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:01.998060 3892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:01.998183 3892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:01.999253 3892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf3b076830404b9745cd50a6a440eb" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39521 } }
I20260809 03:35:01.999541 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:01.981798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37438 (local address 127.1.224.126:39521)
0809 03:35:01.982275 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:01.982302 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:01.982354 (+ 52us) server_negotiation.cc:407] Connection header received
0809 03:35:01.983507 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:01.983537 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:01.984021 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:01.984430 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:01.985430 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:01.986481 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:01.987646 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:01.988125 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:01.988247 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:01.993563 (+ 5316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:01.993569 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:01.993583 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:01.993612 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:01.996266 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:01.996878 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:01.996891 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:01.996896 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:01.997021 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:01.997343 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:01.997347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:01.997350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:01.997798 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:01.998041 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:01.998412 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 03:35:01.998713 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":136,"threads_started":1}
I20260809 03:35:01.999635 3892 leader_election.cc:302] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47bf3b076830404b9745cd50a6a440eb; no voters:
I20260809 03:35:02.000043 3892 leader_election.cc:288] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:02.000123 3906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:02.000332 3906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [term 1 LEADER]: Becoming Leader. State: Replica: 47bf3b076830404b9745cd50a6a440eb, State: Running, Role: LEADER
I20260809 03:35:02.000695 3906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf3b076830404b9745cd50a6a440eb" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39521 } }
I20260809 03:35:02.001367 3892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:02.001765 3906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47bf3b076830404b9745cd50a6a440eb. Latest consensus state: current_term: 1 leader_uuid: "47bf3b076830404b9745cd50a6a440eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf3b076830404b9745cd50a6a440eb" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39521 } } }
I20260809 03:35:02.001880 3906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:02.002331 3911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:02.002291 3908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47bf3b076830404b9745cd50a6a440eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf3b076830404b9745cd50a6a440eb" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39521 } } }
I20260809 03:35:02.002380 3908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:02.005514 3911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:02.011435 3911 catalog_manager.cc:1347] Generated new cluster ID: f7ed5ae0f2954b06bcbe69e44bde9663
I20260809 03:35:02.011521 3911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:02.028208 3911 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:02.029506 3911 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:02.043262 3911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb: Generated new TSK 0
I20260809 03:35:02.044071 3911 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:35:02.083817 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39521
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:02.211385 3943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:02.211655 3943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:02.211699 3943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:02.215864 3943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:02.215934 3943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:02.215974 3943 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:35:02.216077 3943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:02.221560 3943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.224.126:39521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:02.222703 3943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:02.223762 3943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:02.233788 3952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:02.233769 3953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:02.233930 3943 server_base.cc:1034] running on GCE node
W20260809 03:35:02.233769 3956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:02.234586 3943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:02.235221 3943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:02.236382 3943 hybrid_clock.cc:648] HybridClock initialized: now 1786246502236366 us; error 39 us; skew 500 ppm
Aug 09 03:35:02 dist-test-slave-jq4t krb5kdc[3743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246502, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:02.239351 3943 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:35:02.240243 3943 webserver.cc:460] Webserver started at http://127.1.224.65:36125/ using document root <none> and password file <none>
I20260809 03:35:02.240870 3943 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:02.240935 3943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:02.241119 3943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:02.243291 3943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "20967ca359d9461ab4da01d7130d4a52"
format_stamp: "Formatted at 2026-08-09 03:35:02 on dist-test-slave-jq4t"
server_key: "e341f794af26f8dcc41f9f344f363450"
server_key_iv: "a665fddda44f845cc6754a0a8dbf0b96"
server_key_version: "encryptionkey@0"
I20260809 03:35:02.243826 3943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "20967ca359d9461ab4da01d7130d4a52"
format_stamp: "Formatted at 2026-08-09 03:35:02 on dist-test-slave-jq4t"
server_key: "e341f794af26f8dcc41f9f344f363450"
server_key_iv: "a665fddda44f845cc6754a0a8dbf0b96"
server_key_version: "encryptionkey@0"
I20260809 03:35:02.247660 3943 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:35:02.250315 3973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:02.251654 3943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 03:35:02.251781 3943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "20967ca359d9461ab4da01d7130d4a52"
format_stamp: "Formatted at 2026-08-09 03:35:02 on dist-test-slave-jq4t"
server_key: "e341f794af26f8dcc41f9f344f363450"
server_key_iv: "a665fddda44f845cc6754a0a8dbf0b96"
server_key_version: "encryptionkey@0"
I20260809 03:35:02.251895 3943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:02.264374 3943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:02.267580 3943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:02.267803 3943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:02.269438 3943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:02.269490 3943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:02.269558 3943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:02.269593 3943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:35:02 dist-test-slave-jq4t krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246502, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:02.281060 3943 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:41497
I20260809 03:35:02.281145 4174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:41497 every 8 connection(s)
I20260809 03:35:02.282033 3943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:02.282326 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 3943
I20260809 03:35:02.282454 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:02.282804 1921 external_mini_cluster.cc:1428] Setting key c96bddbe850cd2f6ee35b51e651c1e7a
I20260809 03:35:02.284530 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.270395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:36685 (local address 127.1.224.126:39521)
0809 03:35:02.270535 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.270538 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.271618 (+ 1080us) server_negotiation.cc:407] Connection header received
0809 03:35:02.272819 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.272824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.272908 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.272977 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:02.274648 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.275579 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.277023 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.277274 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.277412 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.280127 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:02.280131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:02.280134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:02.280168 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:02.281780 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.282418 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.282424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.282427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.282526 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.283070 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.283073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.283075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.283352 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:02.283460 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.284143 (+ 683us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.284306 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:35:02.285243 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.270957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39521 (local address 127.1.224.65:36685)
0809 03:35:02.271461 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:02.271505 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:02.272558 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:02.273471 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:02.273481 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:02.273828 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:02.274473 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.274489 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.275714 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.275716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.276886 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.276897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.277470 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.277474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.277652 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.278292 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:02.278308 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:02.279950 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:02.281906 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:02.281912 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:02.281926 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:02.282311 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:02.282708 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:02.282712 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:02.282713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:02.282875 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:02.283485 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:02.283491 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:02.283809 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 03:35:02.284025 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 03:35:02.284407 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":144,"threads_started":1}
I20260809 03:35:02.286535 4176 heartbeater.cc:347] Connected to a master server at 127.1.224.126:39521
I20260809 03:35:02.286695 4176 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:02.287299 4176 heartbeater.cc:511] Master 127.1.224.126:39521 requested a full tablet report, sending...
I20260809 03:35:02.289088 3811 ts_manager.cc:194] Registered new tserver with Master: 20967ca359d9461ab4da01d7130d4a52 (127.1.224.65:41497)
I20260809 03:35:02.290666 3811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:36685
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:35:02.296913 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.289293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:37749 (local address 127.1.224.126:39521)
0809 03:35:02.289503 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.289507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.289520 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:02.289554 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.289557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.289594 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.289668 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:02.290281 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.291015 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.292055 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.292283 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.292403 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.293553 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:02.293556 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:02.293559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:02.293583 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:02.295061 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.295596 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.295601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.295603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.295689 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.295997 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.296000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.296002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.296186 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:02.296301 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.296619 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.296783 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260809 03:35:02.296912 4186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.288956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39521 (local address 127.1.224.65:37749)
0809 03:35:02.289286 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:02.289305 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:02.289406 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:02.289783 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:02.289786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:02.290026 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:02.290148 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.290155 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.291184 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.291187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.291928 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.291939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.292414 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.292417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.292535 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.293079 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:02.293091 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:02.293412 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:02.295176 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:02.295182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:02.295187 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:02.295478 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:02.295790 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:02.295793 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:02.295795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:02.295896 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:02.296292 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:02.296295 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:02.296399 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 03:35:02.296541 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 03:35:02.296690 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":106,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:35:02.315258 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39521
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:02.433048 4191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:02.433363 4191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:02.433449 4191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:02.437497 4191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:02.437592 4191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:02.437665 4191 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:35:02.437752 4191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:02.443161 4191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.224.126:39521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:02.444391 4191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:02.445525 4191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:02.456032 4197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:02.456146 4204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:02.456346 4191 server_base.cc:1034] running on GCE node
W20260809 03:35:02.456053 4201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:02.456910 4191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:02.457540 4191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:02.458703 4191 hybrid_clock.cc:648] HybridClock initialized: now 1786246502458670 us; error 47 us; skew 500 ppm
Aug 09 03:35:02 dist-test-slave-jq4t krb5kdc[3743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246502, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:02.461802 4191 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:35:02.462771 4191 webserver.cc:460] Webserver started at http://127.1.224.66:40205/ using document root <none> and password file <none>
I20260809 03:35:02.463325 4191 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:02.463383 4191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:02.463564 4191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:02.465777 4191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "152a093c665a4fc6a7c6b347504ad34f"
format_stamp: "Formatted at 2026-08-09 03:35:02 on dist-test-slave-jq4t"
server_key: "e6c54fa41ac5817986b3c2af10140875"
server_key_iv: "420c59bc2827ccb0808eab0f95f3057a"
server_key_version: "encryptionkey@0"
I20260809 03:35:02.466341 4191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "152a093c665a4fc6a7c6b347504ad34f"
format_stamp: "Formatted at 2026-08-09 03:35:02 on dist-test-slave-jq4t"
server_key: "e6c54fa41ac5817986b3c2af10140875"
server_key_iv: "420c59bc2827ccb0808eab0f95f3057a"
server_key_version: "encryptionkey@0"
I20260809 03:35:02.470289 4191 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:02.472926 4216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:02.474385 4191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:02.474539 4191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "152a093c665a4fc6a7c6b347504ad34f"
format_stamp: "Formatted at 2026-08-09 03:35:02 on dist-test-slave-jq4t"
server_key: "e6c54fa41ac5817986b3c2af10140875"
server_key_iv: "420c59bc2827ccb0808eab0f95f3057a"
server_key_version: "encryptionkey@0"
I20260809 03:35:02.474665 4191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:02.507424 4191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:02.511224 4191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:02.511457 4191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:02.513209 4191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:02.513286 4191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:02.513366 4191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:02.513420 4191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:35:02 dist-test-slave-jq4t krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246502, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:02.525103 4191 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:33323
I20260809 03:35:02.525180 4420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:33323 every 8 connection(s)
I20260809 03:35:02.526310 4191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:02.529413 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.514049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:45735 (local address 127.1.224.126:39521)
0809 03:35:02.514237 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.514243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.515624 (+ 1381us) server_negotiation.cc:407] Connection header received
0809 03:35:02.516949 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.516953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.517020 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.517132 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:02.518984 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.519652 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.521636 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.521888 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.522018 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.525363 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:02.525369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:02.525377 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:02.525408 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:02.527065 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.527729 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.527732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.527734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.527824 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.528173 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.528176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.528177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.528385 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:02.528462 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.529103 (+ 641us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.529283 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 03:35:02.530224 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.514940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39521 (local address 127.1.224.66:45735)
0809 03:35:02.515448 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:02.515505 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:02.516703 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:02.517367 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:02.517379 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:02.517934 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:02.518745 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.518767 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.519807 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.519812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.521487 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.521499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.522073 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.522077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.522278 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.523141 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:02.523169 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:02.525151 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:02.527210 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:02.527215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:02.527254 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:02.527610 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:02.527911 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:02.527913 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:02.527915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:02.528063 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:02.528533 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:02.528542 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:02.528818 (+ 276us) client_negotiation.cc:769] Sending connection context
0809 03:35:02.528999 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 03:35:02.529280 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":153,"threads_started":1}
I20260809 03:35:02.531563 4421 heartbeater.cc:347] Connected to a master server at 127.1.224.126:39521
I20260809 03:35:02.531731 4421 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:02.532371 4421 heartbeater.cc:511] Master 127.1.224.126:39521 requested a full tablet report, sending...
I20260809 03:35:02.533435 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 4191
I20260809 03:35:02.533560 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:02.533694 3812 ts_manager.cc:194] Registered new tserver with Master: 152a093c665a4fc6a7c6b347504ad34f (127.1.224.66:33323)
I20260809 03:35:02.533882 1921 external_mini_cluster.cc:1428] Setting key ccef658e30efab53ac99e8853a3e225f
I20260809 03:35:02.534549 3812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:45735
I20260809 03:35:02.543176 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.534193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39521 (local address 127.1.224.66:60731)
0809 03:35:02.534992 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:02.535008 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:02.535116 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:02.535373 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:02.535376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:02.535616 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:02.535742 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.535750 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.536893 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.536896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.537559 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.537574 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.538049 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.538055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.538182 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.538793 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:02.538815 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:02.539304 (+ 489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:02.541306 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:02.541312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:02.541316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:02.541620 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:02.541911 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:02.541916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:02.541919 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:02.542034 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:02.542518 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:02.542521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:02.542639 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 03:35:02.542755 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 03:35:02.542959 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":121,"threads_started":1}
I20260809 03:35:02.543169 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.534281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:60731 (local address 127.1.224.126:39521)
0809 03:35:02.534423 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.534430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.535092 (+ 662us) server_negotiation.cc:407] Connection header received
0809 03:35:02.535200 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.535204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.535253 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.535336 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:02.535887 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.536747 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.537697 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.537923 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.538004 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.539461 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:02.539465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:02.539467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:02.539493 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:02.541155 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.541726 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.541728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.541729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.541810 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.542137 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.542139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.542140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.542406 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:02.542529 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.542820 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.542969 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:35:02.565450 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39521
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:02.685366 4441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:02.685637 4441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:02.685683 4441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:02.689781 4441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:02.689841 4441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:02.689879 4441 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:35:02.689911 4441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:02.695147 4441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.224.126:39521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:02.696259 4441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:02.697292 4441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:02.707312 4452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:02.707312 4450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:02.707378 4455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:02.707628 4441 server_base.cc:1034] running on GCE node
I20260809 03:35:02.708029 4441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:02.708916 4441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:02.710101 4441 hybrid_clock.cc:648] HybridClock initialized: now 1786246502710030 us; error 83 us; skew 500 ppm
Aug 09 03:35:02 dist-test-slave-jq4t krb5kdc[3743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246502, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:02.713393 4441 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:35:02.714306 4441 webserver.cc:460] Webserver started at http://127.1.224.67:38923/ using document root <none> and password file <none>
I20260809 03:35:02.715036 4441 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:02.715137 4441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:02.715382 4441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:02.717561 4441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4"
format_stamp: "Formatted at 2026-08-09 03:35:02 on dist-test-slave-jq4t"
server_key: "e7537033d1c609532c54d6072c88d73a"
server_key_iv: "781bd6a2dda67b54e4b32cf2963ab59b"
server_key_version: "encryptionkey@0"
I20260809 03:35:02.718156 4441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4"
format_stamp: "Formatted at 2026-08-09 03:35:02 on dist-test-slave-jq4t"
server_key: "e7537033d1c609532c54d6072c88d73a"
server_key_iv: "781bd6a2dda67b54e4b32cf2963ab59b"
server_key_version: "encryptionkey@0"
I20260809 03:35:02.721920 4441 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:35:02.724377 4466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:02.725647 4441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:02.725795 4441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4"
format_stamp: "Formatted at 2026-08-09 03:35:02 on dist-test-slave-jq4t"
server_key: "e7537033d1c609532c54d6072c88d73a"
server_key_iv: "781bd6a2dda67b54e4b32cf2963ab59b"
server_key_version: "encryptionkey@0"
I20260809 03:35:02.725920 4441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:02.742655 4441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:02.745950 4441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:02.746197 4441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:02.747936 4441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:02.748020 4441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:02.748100 4441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:02.748140 4441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:35:02 dist-test-slave-jq4t krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246502, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:02.759902 4441 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:43949
I20260809 03:35:02.759986 4662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:43949 every 8 connection(s)
I20260809 03:35:02.761108 4441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:02.763232 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 4441
I20260809 03:35:02.763365 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:02.763722 1921 external_mini_cluster.cc:1428] Setting key cd795a19fbec2379067efc2d06a2fd10
I20260809 03:35:02.764075 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.748738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:51905 (local address 127.1.224.126:39521)
0809 03:35:02.748883 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.748886 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.750617 (+ 1731us) server_negotiation.cc:407] Connection header received
0809 03:35:02.751972 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.751976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.752049 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.752154 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:02.753652 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.754320 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.755726 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.756131 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.756266 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.759407 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:02.759410 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:02.759412 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:02.759443 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:02.761519 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.762175 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.762179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.762181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.762288 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.762661 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.762664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.762665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.762887 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:02.762969 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.763746 (+ 777us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.763943 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:35:02.765226 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.749920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39521 (local address 127.1.224.67:51905)
0809 03:35:02.750450 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:02.750496 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:02.751689 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:02.752378 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:02.752389 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:02.752957 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:02.753483 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.753498 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.754465 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.754469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.755593 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.755604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.756313 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.756317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.756573 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.757277 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:02.757295 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:02.759195 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:02.761654 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:02.761661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:02.761678 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:02.762061 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:02.762398 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:02.762405 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:02.762411 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:02.762553 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:02.763031 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:02.763040 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:02.763414 (+ 374us) client_negotiation.cc:769] Sending connection context
0809 03:35:02.763663 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 03:35:02.764045 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":160,"threads_started":1}
I20260809 03:35:02.766521 4663 heartbeater.cc:347] Connected to a master server at 127.1.224.126:39521
I20260809 03:35:02.766749 4663 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:02.767282 4663 heartbeater.cc:511] Master 127.1.224.126:39521 requested a full tablet report, sending...
I20260809 03:35:02.768530 3812 ts_manager.cc:194] Registered new tserver with Master: 6d0b2ce59dcd444c8b38fdf2cb8b24c4 (127.1.224.67:43949)
I20260809 03:35:02.769356 3812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:51905
I20260809 03:35:02.776165 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.769044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39521 (local address 127.1.224.67:50335)
0809 03:35:02.769348 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:02.769374 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:02.769486 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:02.769765 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:02.769767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:02.770007 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:02.770157 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.770166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.770938 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.770941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.771710 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.771719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.772154 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.772156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.772228 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.772829 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:02.772841 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:02.773199 (+ 358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:02.774810 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:02.774812 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:02.774814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:02.775039 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:02.775280 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:02.775282 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:02.775283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:02.775360 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:02.775707 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:02.775709 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:02.775786 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 03:35:02.775883 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 03:35:02.776003 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":3,"thread_start_us":127,"threads_started":1}
I20260809 03:35:02.776175 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.769112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:50335 (local address 127.1.224.126:39521)
0809 03:35:02.769253 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.769258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.769433 (+ 175us) server_negotiation.cc:407] Connection header received
0809 03:35:02.769574 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.769577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.769617 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.769730 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:02.770278 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.770790 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.771823 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.772053 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.772145 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.773337 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:02.773340 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:02.773342 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:02.773365 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:02.774702 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.775125 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.775128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.775129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.775201 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.775452 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.775454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.775455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.775627 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:02.775702 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.775958 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.776057 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:35:02.778347 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:35:02 dist-test-slave-jq4t krb5kdc[3743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246502, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:35:02 dist-test-slave-jq4t krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246502, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:02.807036 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.798577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37446 (local address 127.1.224.126:39521)
0809 03:35:02.798716 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.798720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.798731 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:35:02.798776 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.798778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.798819 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.798923 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:02.799675 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.800242 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.801226 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.801462 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.801575 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.804201 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:02.804205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:02.804207 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:02.804234 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:02.805568 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.805962 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.805965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.805967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.806048 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:02.806285 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:02.806287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:02.806288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:02.806470 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:02.806573 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.806789 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.806909 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:35:02.810279 3812 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:02.813479 3812 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:02.829314 4706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.823248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:33323 (local address 127.0.0.1:55148)
0809 03:35:02.823651 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:02.823663 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:02.823738 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:02.824670 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:02.824673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:02.824687 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:02.824785 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.824791 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.826427 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.826432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.827471 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.827482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.828628 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.828632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.828770 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.828818 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 03:35:02.828947 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 03:35:02.829120 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":47,"thread_start_us":99,"threads_started":1}
I20260809 03:35:02.829775 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.823018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35116 (local address 127.1.224.65:41497)
0809 03:35:02.823322 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.823329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.823595 (+ 266us) server_negotiation.cc:407] Connection header received
0809 03:35:02.823832 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.823840 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.824082 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.824207 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:02.824788 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.825827 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.827824 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.828394 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.828552 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.828733 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.829526 (+ 793us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.829631 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":77,"threads_started":1}
I20260809 03:35:02.829798 4697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.822485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:43949 (local address 127.0.0.1:57224)
0809 03:35:02.823180 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:02.823235 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:02.823374 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:02.824510 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:02.824513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:02.824543 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:02.824688 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.824696 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.826299 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.826303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.827525 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.827537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.828628 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.828632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.829068 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.829091 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:35:02.829212 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 03:35:02.829288 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":313,"spinlock_wait_cycles":384,"thread_start_us":101,"threads_started":1}
I20260809 03:35:02.830307 4710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.822599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:41497 (local address 127.0.0.1:35116)
0809 03:35:02.823488 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:02.823509 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:02.823667 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:02.824432 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:02.824440 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:02.824464 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:02.824651 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.824666 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.825992 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.825998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.827704 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.827712 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.828564 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.828567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.828986 (+ 419us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.829348 (+ 362us) client_negotiation.cc:769] Sending connection context
0809 03:35:02.830048 (+ 700us) client_negotiation.cc:240] Negotiation successful
0809 03:35:02.830137 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":593,"spinlock_wait_cycles":1792,"thread_start_us":132,"threads_started":1}
I20260809 03:35:02.829769 4707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.822646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57224 (local address 127.1.224.67:43949)
0809 03:35:02.823994 (+ 1348us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.824005 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.824028 (+ 23us) server_negotiation.cc:407] Connection header received
0809 03:35:02.824114 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.824119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.824328 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.824524 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:02.824829 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.826179 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.827651 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.828450 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.828704 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.828878 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.829253 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.829367 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1247,"thread_start_us":110,"threads_started":1}
I20260809 03:35:02.830413 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.823725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55148 (local address 127.1.224.66:33323)
0809 03:35:02.824215 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.824223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.824242 (+ 19us) server_negotiation.cc:407] Connection header received
0809 03:35:02.824304 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.824308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.824512 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.824933 (+ 421us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:02.825067 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.826283 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.827596 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.828450 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.829520 (+ 1070us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.830095 (+ 575us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.830153 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.830258 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":120,"threads_started":1}
I20260809 03:35:02.833760 4066 tablet_service.cc:1467] Processing CreateTablet for tablet c28a0b8a00e645b0bae8e9fa309c7150 (DEFAULT_TABLE table=test-table [id=4433c50185d842248482340cc7c6b261]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:02.833770 4311 tablet_service.cc:1467] Processing CreateTablet for tablet c28a0b8a00e645b0bae8e9fa309c7150 (DEFAULT_TABLE table=test-table [id=4433c50185d842248482340cc7c6b261]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:02.834502 4557 tablet_service.cc:1467] Processing CreateTablet for tablet c28a0b8a00e645b0bae8e9fa309c7150 (DEFAULT_TABLE table=test-table [id=4433c50185d842248482340cc7c6b261]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:02.835000 4311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c28a0b8a00e645b0bae8e9fa309c7150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:02.835000 4066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c28a0b8a00e645b0bae8e9fa309c7150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:02.835311 4557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c28a0b8a00e645b0bae8e9fa309c7150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:02.842422 4720 tablet_bootstrap.cc:492] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Bootstrap starting.
I20260809 03:35:02.844874 4723 tablet_bootstrap.cc:492] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52: Bootstrap starting.
I20260809 03:35:02.844918 4724 tablet_bootstrap.cc:492] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f: Bootstrap starting.
I20260809 03:35:02.845554 4720 tablet_bootstrap.cc:654] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:02.846554 4720 log.cc:824] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:02.847090 4724 tablet_bootstrap.cc:654] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:02.847199 4723 tablet_bootstrap.cc:654] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:02.847879 4724 log.cc:824] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:02.848008 4723 log.cc:824] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:02.849011 4720 tablet_bootstrap.cc:492] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: No bootstrap required, opened a new log
I20260809 03:35:02.849242 4720 ts_tablet_manager.cc:1397] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:35:02.849870 4724 tablet_bootstrap.cc:492] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f: No bootstrap required, opened a new log
I20260809 03:35:02.850092 4724 ts_tablet_manager.cc:1397] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:35:02.850764 4723 tablet_bootstrap.cc:492] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52: No bootstrap required, opened a new log
I20260809 03:35:02.850975 4723 ts_tablet_manager.cc:1397] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:35:02.853564 4720 raft_consensus.cc:348] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } }
I20260809 03:35:02.853859 4720 raft_consensus.cc:374] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:02.853931 4720 raft_consensus.cc:729] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0b2ce59dcd444c8b38fdf2cb8b24c4, State: Initialized, Role: FOLLOWER
I20260809 03:35:02.853765 4724 raft_consensus.cc:348] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } }
I20260809 03:35:02.854022 4724 raft_consensus.cc:374] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:02.854084 4724 raft_consensus.cc:729] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152a093c665a4fc6a7c6b347504ad34f, State: Initialized, Role: FOLLOWER
I20260809 03:35:02.854460 4720 consensus_queue.cc:260] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } }
I20260809 03:35:02.854506 4724 consensus_queue.cc:260] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } }
I20260809 03:35:02.855496 4663 heartbeater.cc:503] Master 127.1.224.126:39521 was elected leader, sending a full tablet report...
I20260809 03:35:02.855600 4724 ts_tablet_manager.cc:1428] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 03:35:02.855614 4720 ts_tablet_manager.cc:1428] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:35:02.855443 4723 raft_consensus.cc:348] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } }
I20260809 03:35:02.855687 4723 raft_consensus.cc:374] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:02.855732 4723 raft_consensus.cc:729] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20967ca359d9461ab4da01d7130d4a52, State: Initialized, Role: FOLLOWER
I20260809 03:35:02.856060 4421 heartbeater.cc:503] Master 127.1.224.126:39521 was elected leader, sending a full tablet report...
I20260809 03:35:02.856242 4723 consensus_queue.cc:260] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } }
I20260809 03:35:02.857383 4723 ts_tablet_manager.cc:1428] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260809 03:35:02.857595 4176 heartbeater.cc:503] Master 127.1.224.126:39521 was elected leader, sending a full tablet report...
I20260809 03:35:02.935992 4733 raft_consensus.cc:482] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:02.936241 4733 raft_consensus.cc:504] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } }
I20260809 03:35:02.937577 4733 leader_election.cc:288] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20967ca359d9461ab4da01d7130d4a52 (127.1.224.65:41497), 152a093c665a4fc6a7c6b347504ad34f (127.1.224.66:33323)
I20260809 03:35:02.941537 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.937911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:41497 (local address 127.1.224.67:54547)
0809 03:35:02.938058 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:02.938077 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:02.938188 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:02.938507 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:02.938510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:02.938530 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:02.938669 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.938675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.939467 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.939474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.940297 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.940306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.941069 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.941072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.941177 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.941201 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:35:02.941304 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 03:35:02.941386 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 03:35:02.941537 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.938037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:54547 (local address 127.1.224.65:41497)
0809 03:35:02.938206 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.938212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.938234 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:35:02.938307 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.938311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.938369 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.938472 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:02.938801 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.939349 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.940446 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.940934 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.941077 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.941140 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.941323 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.941403 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:35:02.942369 4105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28a0b8a00e645b0bae8e9fa309c7150" candidate_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20967ca359d9461ab4da01d7130d4a52" is_pre_election: true
I20260809 03:35:02.942677 4735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.938230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:33323 (local address 127.1.224.67:49165)
0809 03:35:02.938724 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:02.938735 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:02.938857 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:02.939141 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:02.939144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:02.939162 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:02.939297 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.939302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.940043 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.940049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.941356 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:02.941367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.942297 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.942301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:02.942374 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.942394 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 03:35:02.942454 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 03:35:02.942509 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":163,"threads_started":1}
I20260809 03:35:02.942716 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.938415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:49165 (local address 127.1.224.66:33323)
0809 03:35:02.938558 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.938564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.938822 (+ 258us) server_negotiation.cc:407] Connection header received
0809 03:35:02.938988 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.938990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.939034 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.939125 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:02.939409 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.939915 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.941499 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.942156 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.942283 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.942331 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.942506 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.942593 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:35:02.942812 4105 raft_consensus.cc:2454] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0b2ce59dcd444c8b38fdf2cb8b24c4 in term 0.
I20260809 03:35:02.943333 4353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28a0b8a00e645b0bae8e9fa309c7150" candidate_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152a093c665a4fc6a7c6b347504ad34f" is_pre_election: true
I20260809 03:35:02.943360 4471 leader_election.cc:302] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20967ca359d9461ab4da01d7130d4a52, 6d0b2ce59dcd444c8b38fdf2cb8b24c4; no voters:
I20260809 03:35:02.943653 4353 raft_consensus.cc:2454] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d0b2ce59dcd444c8b38fdf2cb8b24c4 in term 0.
I20260809 03:35:02.943681 4733 raft_consensus.cc:2781] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:02.943766 4733 raft_consensus.cc:482] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:02.943838 4733 raft_consensus.cc:3037] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:02.945242 4733 raft_consensus.cc:504] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } }
I20260809 03:35:02.945647 4733 leader_election.cc:288] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [CANDIDATE]: Term 1 election: Requested vote from peers 20967ca359d9461ab4da01d7130d4a52 (127.1.224.65:41497), 152a093c665a4fc6a7c6b347504ad34f (127.1.224.66:33323)
I20260809 03:35:02.946058 4353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28a0b8a00e645b0bae8e9fa309c7150" candidate_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152a093c665a4fc6a7c6b347504ad34f"
I20260809 03:35:02.946165 4105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c28a0b8a00e645b0bae8e9fa309c7150" candidate_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20967ca359d9461ab4da01d7130d4a52"
I20260809 03:35:02.946182 4353 raft_consensus.cc:3037] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:02.946322 4105 raft_consensus.cc:3037] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:02.947397 4105 raft_consensus.cc:2454] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0b2ce59dcd444c8b38fdf2cb8b24c4 in term 1.
I20260809 03:35:02.947722 4471 leader_election.cc:302] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20967ca359d9461ab4da01d7130d4a52, 6d0b2ce59dcd444c8b38fdf2cb8b24c4; no voters:
I20260809 03:35:02.947909 4733 raft_consensus.cc:2781] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:02.947904 4353 raft_consensus.cc:2454] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d0b2ce59dcd444c8b38fdf2cb8b24c4 in term 1.
I20260809 03:35:02.948176 4733 raft_consensus.cc:686] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 1 LEADER]: Becoming Leader. State: Replica: 6d0b2ce59dcd444c8b38fdf2cb8b24c4, State: Running, Role: LEADER
I20260809 03:35:02.948585 4733 consensus_queue.cc:237] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } }
I20260809 03:35:02.951740 3809 catalog_manager.cc:5495] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d0b2ce59dcd444c8b38fdf2cb8b24c4 (127.1.224.67). New cstate: current_term: 1 leader_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } health_report { overall_health: UNKNOWN } } }
I20260809 03:35:02.993461 4707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:02.989637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57232 (local address 127.1.224.67:43949)
0809 03:35:02.989778 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:02.989782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:02.989796 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:35:02.989996 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:02.989998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:02.990062 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:02.990156 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:02.990421 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.990987 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.991863 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:02.992091 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:02.992169 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:02.992559 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:02.993044 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:02.993141 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:02.993271 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 03:35:02.993333 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:35:03.000418 4353 raft_consensus.cc:1261] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f [term 1 FOLLOWER]: Refusing update from remote peer 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:03.001370 4733 consensus_queue.cc:1035] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:03.001581 4105 raft_consensus.cc:1261] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52 [term 1 FOLLOWER]: Refusing update from remote peer 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:03.002178 4733 consensus_queue.cc:1035] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:03.011927 4743 mvcc.cc:205] Tried to move back new op lower bound from 7316465676280205312 to 7316465676080144384. Current Snapshot: MvccSnapshot[applied={T|T < 7316465676280205312}]
I20260809 03:35:03.013238 4742 mvcc.cc:205] Tried to move back new op lower bound from 7316465676280205312 to 7316465676080144384. Current Snapshot: MvccSnapshot[applied={T|T < 7316465676280205312}]
I20260809 03:35:03.023722 3809 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:03.023962 3809 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:03.027114 3809 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb: Sending DeleteTablet for 3 replicas of tablet c28a0b8a00e645b0bae8e9fa309c7150
I20260809 03:35:03.028139 4557 tablet_service.cc:1514] Processing DeleteTablet for tablet c28a0b8a00e645b0bae8e9fa309c7150 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:03 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:57224
I20260809 03:35:03.028507 4311 tablet_service.cc:1514] Processing DeleteTablet for tablet c28a0b8a00e645b0bae8e9fa309c7150 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:03 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:55148
I20260809 03:35:03.028802 4066 tablet_service.cc:1514] Processing DeleteTablet for tablet c28a0b8a00e645b0bae8e9fa309c7150 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:03 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:35116
I20260809 03:35:03.029153 4757 tablet_replica.cc:330] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f: stopping tablet replica
I20260809 03:35:03.029444 4756 tablet_replica.cc:330] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: stopping tablet replica
I20260809 03:35:03.029556 4757 raft_consensus.cc:2229] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:03.029762 4756 raft_consensus.cc:2229] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:03.029951 4757 raft_consensus.cc:2258] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:03.030244 4756 raft_consensus.cc:2258] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:03.030404 4759 tablet_replica.cc:330] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52: stopping tablet replica
I20260809 03:35:03.030444 3809 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 03:35:03.030719 4759 raft_consensus.cc:2229] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:03.031064 4759 raft_consensus.cc:2258] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 03:35:03.031039 3809 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:03.032244 4757 ts_tablet_manager.cc:1905] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:03.033270 4756 ts_tablet_manager.cc:1905] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:03.035488 4757 ts_tablet_manager.cc:1918] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:03.035605 4757 log.cc:1196] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/c28a0b8a00e645b0bae8e9fa309c7150
I20260809 03:35:03.035934 4757 ts_tablet_manager.cc:1939] T c28a0b8a00e645b0bae8e9fa309c7150 P 152a093c665a4fc6a7c6b347504ad34f: Deleting consensus metadata
I20260809 03:35:03.036401 4756 ts_tablet_manager.cc:1918] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:03.036554 4756 log.cc:1196] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/c28a0b8a00e645b0bae8e9fa309c7150
I20260809 03:35:03.036926 4756 ts_tablet_manager.cc:1939] T c28a0b8a00e645b0bae8e9fa309c7150 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Deleting consensus metadata
I20260809 03:35:03.037372 3789 catalog_manager.cc:4841] TS 152a093c665a4fc6a7c6b347504ad34f (127.1.224.66:33323): tablet c28a0b8a00e645b0bae8e9fa309c7150 (table test-table [id=4433c50185d842248482340cc7c6b261]) successfully deleted
I20260809 03:35:03.038203 3790 catalog_manager.cc:4841] TS 6d0b2ce59dcd444c8b38fdf2cb8b24c4 (127.1.224.67:43949): tablet c28a0b8a00e645b0bae8e9fa309c7150 (table test-table [id=4433c50185d842248482340cc7c6b261]) successfully deleted
I20260809 03:35:03.038673 4759 ts_tablet_manager.cc:1905] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:03.038693 4557 tablet_service.cc:1467] Processing CreateTablet for tablet b268f988778048d08f474fe6366e519c (DEFAULT_TABLE table=test-table [id=4f61259a40184f66aafb4126a1b070a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:03.038980 4557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b268f988778048d08f474fe6366e519c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:03.039065 4311 tablet_service.cc:1467] Processing CreateTablet for tablet b268f988778048d08f474fe6366e519c (DEFAULT_TABLE table=test-table [id=4f61259a40184f66aafb4126a1b070a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:03.039304 4311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b268f988778048d08f474fe6366e519c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:03.041407 4759 ts_tablet_manager.cc:1918] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:03.041429 4720 tablet_bootstrap.cc:492] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Bootstrap starting.
I20260809 03:35:03.041504 4759 log.cc:1196] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/wal/wals/c28a0b8a00e645b0bae8e9fa309c7150
I20260809 03:35:03.041806 4759 ts_tablet_manager.cc:1939] T c28a0b8a00e645b0bae8e9fa309c7150 P 20967ca359d9461ab4da01d7130d4a52: Deleting consensus metadata
I20260809 03:35:03.041818 4724 tablet_bootstrap.cc:492] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f: Bootstrap starting.
I20260809 03:35:03.042444 4720 tablet_bootstrap.cc:654] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:03.042809 4724 tablet_bootstrap.cc:654] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:03.038693 4066 tablet_service.cc:1467] Processing CreateTablet for tablet b268f988778048d08f474fe6366e519c (DEFAULT_TABLE table=test-table [id=4f61259a40184f66aafb4126a1b070a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:03.043395 4066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b268f988778048d08f474fe6366e519c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:03.043507 3789 catalog_manager.cc:4841] TS 20967ca359d9461ab4da01d7130d4a52 (127.1.224.65:41497): tablet c28a0b8a00e645b0bae8e9fa309c7150 (table test-table [id=4433c50185d842248482340cc7c6b261]) successfully deleted
I20260809 03:35:03.044441 4720 tablet_bootstrap.cc:492] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: No bootstrap required, opened a new log
I20260809 03:35:03.044436 4724 tablet_bootstrap.cc:492] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f: No bootstrap required, opened a new log
I20260809 03:35:03.044551 4720 ts_tablet_manager.cc:1397] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260809 03:35:03.044551 4724 ts_tablet_manager.cc:1397] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:03.044992 4720 raft_consensus.cc:348] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.044992 4724 raft_consensus.cc:348] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.045102 4720 raft_consensus.cc:374] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:03.045102 4724 raft_consensus.cc:374] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:03.045136 4724 raft_consensus.cc:729] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152a093c665a4fc6a7c6b347504ad34f, State: Initialized, Role: FOLLOWER
I20260809 03:35:03.045136 4720 raft_consensus.cc:729] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0b2ce59dcd444c8b38fdf2cb8b24c4, State: Initialized, Role: FOLLOWER
I20260809 03:35:03.045272 4724 consensus_queue.cc:260] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.045269 4720 consensus_queue.cc:260] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.045527 4720 ts_tablet_manager.cc:1428] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:03.045527 4724 ts_tablet_manager.cc:1428] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:03.045890 4723 tablet_bootstrap.cc:492] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52: Bootstrap starting.
I20260809 03:35:03.046911 4723 tablet_bootstrap.cc:654] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:03.048286 4723 tablet_bootstrap.cc:492] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52: No bootstrap required, opened a new log
I20260809 03:35:03.048410 4723 ts_tablet_manager.cc:1397] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 03:35:03.049134 4723 raft_consensus.cc:348] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.049230 4723 raft_consensus.cc:374] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:03.049252 4723 raft_consensus.cc:729] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20967ca359d9461ab4da01d7130d4a52, State: Initialized, Role: FOLLOWER
I20260809 03:35:03.049413 4723 consensus_queue.cc:260] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.049682 4723 ts_tablet_manager.cc:1428] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:03.122090 4734 raft_consensus.cc:482] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:03.122253 4734 raft_consensus.cc:504] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.123420 4734 leader_election.cc:288] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 152a093c665a4fc6a7c6b347504ad34f (127.1.224.66:33323), 6d0b2ce59dcd444c8b38fdf2cb8b24c4 (127.1.224.67:43949)
I20260809 03:35:03.128989 4707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.123837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:43731 (local address 127.1.224.67:43949)
0809 03:35:03.124065 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:03.124072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:03.124374 (+ 302us) server_negotiation.cc:407] Connection header received
0809 03:35:03.124582 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:03.124586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:03.124638 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:03.124691 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:03.125071 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.125558 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.127540 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.128208 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.128342 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.128403 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:03.128709 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 03:35:03.128797 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260809 03:35:03.129745 4768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.123706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:43949 (local address 127.1.224.65:43731)
0809 03:35:03.124251 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:03.124266 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:03.124402 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:03.124843 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:03.124844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:03.124859 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:03.124953 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.124957 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.125677 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.125682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.127387 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.127400 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.128362 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.128366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.128533 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.128564 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 03:35:03.129442 (+ 878us) client_negotiation.cc:240] Negotiation successful
0809 03:35:03.129581 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":158,"thread_start_us":129,"threads_started":1}
I20260809 03:35:03.130012 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.124117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:33513 (local address 127.1.224.66:33323)
0809 03:35:03.124244 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:03.124248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:03.124264 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:35:03.124328 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:03.124331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:03.124381 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:03.124470 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:03.124820 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.125633 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.128503 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.129169 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.129290 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.129339 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:03.129756 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 03:35:03.129863 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:35:03.129993 4767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.123696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:33323 (local address 127.1.224.65:33513)
0809 03:35:03.124040 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:03.124058 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:03.124177 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:03.124511 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:03.124517 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:03.124539 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:03.124684 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.124691 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.125728 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.125733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.127387 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.127400 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.129436 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.129440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.129602 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.129630 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 03:35:03.129712 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 03:35:03.129786 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":201,"thread_start_us":92,"threads_started":1}
I20260809 03:35:03.130757 4353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b268f988778048d08f474fe6366e519c" candidate_uuid: "20967ca359d9461ab4da01d7130d4a52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152a093c665a4fc6a7c6b347504ad34f" is_pre_election: true
I20260809 03:35:03.130755 4589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b268f988778048d08f474fe6366e519c" candidate_uuid: "20967ca359d9461ab4da01d7130d4a52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" is_pre_election: true
I20260809 03:35:03.130992 4353 raft_consensus.cc:2454] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20967ca359d9461ab4da01d7130d4a52 in term 0.
I20260809 03:35:03.131094 4589 raft_consensus.cc:2454] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20967ca359d9461ab4da01d7130d4a52 in term 0.
I20260809 03:35:03.131451 3976 leader_election.cc:302] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152a093c665a4fc6a7c6b347504ad34f, 20967ca359d9461ab4da01d7130d4a52; no voters:
I20260809 03:35:03.131974 4734 raft_consensus.cc:2781] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:03.132033 4734 raft_consensus.cc:482] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:03.132061 4734 raft_consensus.cc:3037] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:03.133339 4734 raft_consensus.cc:504] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.133771 4734 leader_election.cc:288] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [CANDIDATE]: Term 1 election: Requested vote from peers 152a093c665a4fc6a7c6b347504ad34f (127.1.224.66:33323), 6d0b2ce59dcd444c8b38fdf2cb8b24c4 (127.1.224.67:43949)
I20260809 03:35:03.134194 4589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b268f988778048d08f474fe6366e519c" candidate_uuid: "20967ca359d9461ab4da01d7130d4a52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4"
I20260809 03:35:03.134179 4353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b268f988778048d08f474fe6366e519c" candidate_uuid: "20967ca359d9461ab4da01d7130d4a52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "152a093c665a4fc6a7c6b347504ad34f"
I20260809 03:35:03.134307 4589 raft_consensus.cc:3037] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:03.134341 4353 raft_consensus.cc:3037] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:03.135582 4353 raft_consensus.cc:2454] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20967ca359d9461ab4da01d7130d4a52 in term 1.
I20260809 03:35:03.135797 4589 raft_consensus.cc:2454] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20967ca359d9461ab4da01d7130d4a52 in term 1.
I20260809 03:35:03.135943 3976 leader_election.cc:302] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152a093c665a4fc6a7c6b347504ad34f, 20967ca359d9461ab4da01d7130d4a52; no voters:
I20260809 03:35:03.136168 4734 raft_consensus.cc:2781] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:03.136396 4734 raft_consensus.cc:686] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 1 LEADER]: Becoming Leader. State: Replica: 20967ca359d9461ab4da01d7130d4a52, State: Running, Role: LEADER
I20260809 03:35:03.136725 4734 consensus_queue.cc:237] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.139526 3812 catalog_manager.cc:5495] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20967ca359d9461ab4da01d7130d4a52 (127.1.224.65). New cstate: current_term: 1 leader_uuid: "20967ca359d9461ab4da01d7130d4a52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } health_report { overall_health: UNKNOWN } } }
Aug 09 03:35:03 dist-test-slave-jq4t krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246501, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:03.184612 4780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.174664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39521 (local address 127.0.0.1:37450)
0809 03:35:03.175029 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:03.175042 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:03.175203 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:03.175476 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:03.175479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:03.175946 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:03.176052 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.176059 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.176972 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.176975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.177595 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.177605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.178130 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.178134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.178249 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.178919 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:03.178935 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:03.180864 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:03.182949 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:03.182960 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:03.182967 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:03.183220 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:03.183508 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:03.183510 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:03.183511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:03.183595 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:03.184019 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:03.184025 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:03.184159 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 03:35:03.184254 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 03:35:03.184388 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":140,"threads_started":1}
I20260809 03:35:03.184665 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.174732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37450 (local address 127.1.224.126:39521)
0809 03:35:03.174899 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:03.174903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:03.175127 (+ 224us) server_negotiation.cc:407] Connection header received
0809 03:35:03.175312 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:03.175323 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:03.175361 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:03.175456 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:03.176189 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.176840 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.177726 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.177947 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.178075 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.181068 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:03.181073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:03.181076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:03.181104 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:03.182806 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:03.183319 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:03.183323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:03.183325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:03.183413 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:03.183679 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:03.183682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:03.183684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:03.183917 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:03.184012 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:03.184333 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 03:35:03.184465 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:35:03.189770 3812 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:37450:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 03:35:03.190558 3812 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:03.195820 4066 tablet_service.cc:1467] Processing CreateTablet for tablet 9dd97d24040248199605ee97ed2145e2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=acfa209aec284f18a422a17e58df97ab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:35:03.195820 4311 tablet_service.cc:1467] Processing CreateTablet for tablet 9dd97d24040248199605ee97ed2145e2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=acfa209aec284f18a422a17e58df97ab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:35:03.195915 4557 tablet_service.cc:1467] Processing CreateTablet for tablet 9dd97d24040248199605ee97ed2145e2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=acfa209aec284f18a422a17e58df97ab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:35:03.196277 4311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dd97d24040248199605ee97ed2145e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:03.196277 4066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dd97d24040248199605ee97ed2145e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:03.196447 4557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dd97d24040248199605ee97ed2145e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:03.199057 4720 tablet_bootstrap.cc:492] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Bootstrap starting.
I20260809 03:35:03.199354 4723 tablet_bootstrap.cc:492] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52: Bootstrap starting.
I20260809 03:35:03.199357 4724 tablet_bootstrap.cc:492] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: Bootstrap starting.
I20260809 03:35:03.200064 4720 tablet_bootstrap.cc:654] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:03.200384 4724 tablet_bootstrap.cc:654] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:03.200433 4723 tablet_bootstrap.cc:654] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:03.201489 4720 tablet_bootstrap.cc:492] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: No bootstrap required, opened a new log
I20260809 03:35:03.201593 4720 ts_tablet_manager.cc:1397] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 03:35:03.201794 4724 tablet_bootstrap.cc:492] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: No bootstrap required, opened a new log
I20260809 03:35:03.201858 4723 tablet_bootstrap.cc:492] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52: No bootstrap required, opened a new log
I20260809 03:35:03.201910 4724 ts_tablet_manager.cc:1397] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 03:35:03.201967 4723 ts_tablet_manager.cc:1397] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 03:35:03.202117 4720 raft_consensus.cc:348] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.202244 4720 raft_consensus.cc:374] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:03.202286 4720 raft_consensus.cc:729] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0b2ce59dcd444c8b38fdf2cb8b24c4, State: Initialized, Role: FOLLOWER
I20260809 03:35:03.202420 4720 consensus_queue.cc:260] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.202471 4723 raft_consensus.cc:348] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.202482 4724 raft_consensus.cc:348] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.202590 4723 raft_consensus.cc:374] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:03.202620 4724 raft_consensus.cc:374] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:03.202625 4723 raft_consensus.cc:729] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20967ca359d9461ab4da01d7130d4a52, State: Initialized, Role: FOLLOWER
I20260809 03:35:03.202669 4724 raft_consensus.cc:729] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152a093c665a4fc6a7c6b347504ad34f, State: Initialized, Role: FOLLOWER
I20260809 03:35:03.202749 4720 ts_tablet_manager.cc:1428] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260809 03:35:03.202761 4723 consensus_queue.cc:260] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.202816 4724 consensus_queue.cc:260] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.203118 4724 ts_tablet_manager.cc:1428] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:03.203039 4736 tablet_replica.cc:439] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } }
I20260809 03:35:03.203056 4734 tablet_replica.cc:439] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } }
I20260809 03:35:03.203205 4736 tablet_replica.cc:442] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:03.203225 4734 tablet_replica.cc:442] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:03.203276 4732 tablet_replica.cc:439] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } }
I20260809 03:35:03.203404 4732 tablet_replica.cc:442] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:03.203070 4723 ts_tablet_manager.cc:1428] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260809 03:35:03.245225 4732 raft_consensus.cc:482] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:03.245409 4732 raft_consensus.cc:504] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.246626 4732 leader_election.cc:288] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20967ca359d9461ab4da01d7130d4a52 (127.1.224.65:41497), 6d0b2ce59dcd444c8b38fdf2cb8b24c4 (127.1.224.67:43949)
I20260809 03:35:03.250841 4707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.247058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:32887 (local address 127.1.224.67:43949)
0809 03:35:03.247204 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:03.247209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:03.247386 (+ 177us) server_negotiation.cc:407] Connection header received
0809 03:35:03.247527 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:03.247530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:03.247573 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:03.247659 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:03.247995 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.248616 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.249822 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.250229 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.250350 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.250412 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:03.250610 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 03:35:03.250690 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 03:35:03.250860 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.247074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:50661 (local address 127.1.224.65:41497)
0809 03:35:03.247219 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:03.247224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:03.247345 (+ 121us) server_negotiation.cc:407] Connection header received
0809 03:35:03.247528 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:03.247530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:03.247574 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:03.247658 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:03.248036 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.248629 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.249857 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.250248 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.250381 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.250446 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:03.250658 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 03:35:03.250738 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:35:03.250874 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.246864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:41497 (local address 127.1.224.66:50661)
0809 03:35:03.247236 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:03.247260 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:03.247387 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:03.247689 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:03.247692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:03.247726 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:03.247894 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.247900 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.248759 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.248763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.249698 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.249708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.250398 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.250402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.250510 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.250527 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 03:35:03.250625 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 03:35:03.250689 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":85,"spinlock_wait_cycles":10496,"thread_start_us":118,"threads_started":1}
I20260809 03:35:03.250882 4792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.246892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:43949 (local address 127.1.224.66:32887)
0809 03:35:03.247256 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:03.247278 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:03.247412 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:03.247685 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:03.247688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:03.247708 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:03.247878 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.247884 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.248726 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.248731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.249689 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.249698 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.250356 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.250360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.250470 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.250500 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 03:35:03.250561 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 03:35:03.250648 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":43,"spinlock_wait_cycles":30592,"thread_start_us":95,"threads_started":1}
I20260809 03:35:03.251368 4105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dd97d24040248199605ee97ed2145e2" candidate_uuid: "152a093c665a4fc6a7c6b347504ad34f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20967ca359d9461ab4da01d7130d4a52" is_pre_election: true
I20260809 03:35:03.251555 4105 raft_consensus.cc:2454] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152a093c665a4fc6a7c6b347504ad34f in term 0.
I20260809 03:35:03.251596 4589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dd97d24040248199605ee97ed2145e2" candidate_uuid: "152a093c665a4fc6a7c6b347504ad34f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" is_pre_election: true
I20260809 03:35:03.251775 4589 raft_consensus.cc:2454] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152a093c665a4fc6a7c6b347504ad34f in term 0.
I20260809 03:35:03.251986 4221 leader_election.cc:302] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152a093c665a4fc6a7c6b347504ad34f, 20967ca359d9461ab4da01d7130d4a52; no voters:
I20260809 03:35:03.252269 4732 raft_consensus.cc:2781] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:03.252338 4732 raft_consensus.cc:482] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:03.252404 4732 raft_consensus.cc:3037] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:03.253643 4732 raft_consensus.cc:504] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.254041 4732 leader_election.cc:288] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [CANDIDATE]: Term 1 election: Requested vote from peers 20967ca359d9461ab4da01d7130d4a52 (127.1.224.65:41497), 6d0b2ce59dcd444c8b38fdf2cb8b24c4 (127.1.224.67:43949)
I20260809 03:35:03.254367 4105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dd97d24040248199605ee97ed2145e2" candidate_uuid: "152a093c665a4fc6a7c6b347504ad34f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20967ca359d9461ab4da01d7130d4a52"
I20260809 03:35:03.254431 4589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dd97d24040248199605ee97ed2145e2" candidate_uuid: "152a093c665a4fc6a7c6b347504ad34f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4"
I20260809 03:35:03.254510 4105 raft_consensus.cc:3037] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:03.254563 4589 raft_consensus.cc:3037] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:03.255553 4105 raft_consensus.cc:2454] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152a093c665a4fc6a7c6b347504ad34f in term 1.
I20260809 03:35:03.255590 4589 raft_consensus.cc:2454] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152a093c665a4fc6a7c6b347504ad34f in term 1.
I20260809 03:35:03.255880 4221 leader_election.cc:302] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152a093c665a4fc6a7c6b347504ad34f, 20967ca359d9461ab4da01d7130d4a52; no voters:
I20260809 03:35:03.256085 4732 raft_consensus.cc:2781] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:03.256300 4732 raft_consensus.cc:686] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [term 1 LEADER]: Becoming Leader. State: Replica: 152a093c665a4fc6a7c6b347504ad34f, State: Running, Role: LEADER
I20260809 03:35:03.256615 4732 consensus_queue.cc:237] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } }
I20260809 03:35:03.258165 4793 tablet_replica.cc:439] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: TxnStatusTablet state changed. Reason: New leader 152a093c665a4fc6a7c6b347504ad34f. Latest consensus state: current_term: 1 leader_uuid: "152a093c665a4fc6a7c6b347504ad34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } }
I20260809 03:35:03.258315 4793 tablet_replica.cc:442] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:35:03.258697 4796 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 03:35:03.258908 4796 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 03:35:03.259641 3812 catalog_manager.cc:5495] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f reported cstate change: term changed from 0 to 1, leader changed from <none> to 152a093c665a4fc6a7c6b347504ad34f (127.1.224.66). New cstate: current_term: 1 leader_uuid: "152a093c665a4fc6a7c6b347504ad34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } health_report { overall_health: UNKNOWN } } }
W20260809 03:35:03.265244 4665 tablet.cc:2369] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:03.275463 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.271189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55164 (local address 127.1.224.66:33323)
0809 03:35:03.271319 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:03.271323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:03.271382 (+ 59us) server_negotiation.cc:407] Connection header received
0809 03:35:03.271604 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:03.271606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:03.271664 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:03.271750 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:03.272058 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.272632 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.273892 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.274146 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.274230 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.274597 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:03.275067 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:03.275159 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:03.275283 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 03:35:03.275341 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:35:03.275518 4780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.271056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:33323 (local address 127.0.0.1:55164)
0809 03:35:03.271314 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:03.271330 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:03.271456 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:03.271798 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:03.271801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:03.271810 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:35:03.271943 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.271950 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.272744 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.272748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.273755 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.273768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.274264 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.274268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.274364 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.274432 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:35:03.275168 (+ 736us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:35:03.275184 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 03:35:03.275263 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 03:35:03.275362 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
W20260809 03:35:03.278453 4422 tablet.cc:2369] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:03.279230 4105 raft_consensus.cc:1261] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52 [term 1 FOLLOWER]: Refusing update from remote peer 152a093c665a4fc6a7c6b347504ad34f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:03.279572 4589 raft_consensus.cc:1261] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 1 FOLLOWER]: Refusing update from remote peer 152a093c665a4fc6a7c6b347504ad34f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:03.279791 4793 consensus_queue.cc:1035] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:03.280402 4793 consensus_queue.cc:1035] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:03.282070 4734 tablet_replica.cc:439] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52: TxnStatusTablet state changed. Reason: New leader 152a093c665a4fc6a7c6b347504ad34f. Latest consensus state: current_term: 1 leader_uuid: "152a093c665a4fc6a7c6b347504ad34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } }
I20260809 03:35:03.282197 4734 tablet_replica.cc:442] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:03.283479 4793 tablet_replica.cc:439] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "152a093c665a4fc6a7c6b347504ad34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } }
I20260809 03:35:03.283589 4793 tablet_replica.cc:442] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: This TxnStatusTablet replica's current role is: LEADER
W20260809 03:35:03.284067 4179 tablet.cc:2369] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:03.284453 4734 tablet_replica.cc:439] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "152a093c665a4fc6a7c6b347504ad34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } }
I20260809 03:35:03.284590 4734 tablet_replica.cc:442] T 9dd97d24040248199605ee97ed2145e2 P 20967ca359d9461ab4da01d7130d4a52: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:03.286255 4736 tablet_replica.cc:439] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: TxnStatusTablet state changed. Reason: New leader 152a093c665a4fc6a7c6b347504ad34f. Latest consensus state: current_term: 1 leader_uuid: "152a093c665a4fc6a7c6b347504ad34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } }
I20260809 03:35:03.286382 4736 tablet_replica.cc:442] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:03.294377 4793 tablet_replica.cc:439] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "152a093c665a4fc6a7c6b347504ad34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } }
I20260809 03:35:03.294552 4793 tablet_replica.cc:442] T 9dd97d24040248199605ee97ed2145e2 P 152a093c665a4fc6a7c6b347504ad34f: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:35:03.295063 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.290798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35120 (local address 127.1.224.65:41497)
0809 03:35:03.290950 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:03.290960 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:03.290971 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:35:03.291037 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:03.291039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:03.291090 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:03.291211 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:03.291461 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.291993 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.293040 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.293361 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.293494 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.293790 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:03.294393 (+ 603us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:03.294523 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:03.294869 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 03:35:03.294934 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 03:35:03.296128 4740 mvcc.cc:205] Tried to move back new op lower bound from 7316465677426016256 to 7316465677341704192. Current Snapshot: MvccSnapshot[applied={T|T < 7316465677426016256}]
I20260809 03:35:03.296902 4736 tablet_replica.cc:439] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "152a093c665a4fc6a7c6b347504ad34f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20967ca359d9461ab4da01d7130d4a52" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 41497 } } peers { permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 } } peers { permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 } } }
I20260809 03:35:03.297037 4736 tablet_replica.cc:442] T 9dd97d24040248199605ee97ed2145e2 P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:03.309185 4810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.305431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:33323 (local address 127.1.224.65:32927)
0809 03:35:03.305739 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:03.305752 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:03.305844 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:03.306087 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:03.306089 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:03.306094 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:35:03.306187 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.306191 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.307161 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.307163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.307791 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.307799 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.308393 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.308396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.308460 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.308608 (+ 148us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:35:03.308917 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:35:03.308922 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:35:03.308968 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 03:35:03.309048 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":119,"threads_started":1}
I20260809 03:35:03.309188 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.305562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:32927 (local address 127.1.224.66:33323)
0809 03:35:03.305691 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:03.305696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:03.305831 (+ 135us) server_negotiation.cc:407] Connection header received
0809 03:35:03.305919 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:03.305922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:03.305995 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:03.306086 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:03.306249 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.307051 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.307914 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.308276 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.308384 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.308737 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:03.308829 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:03.308905 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:03.308993 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 03:35:03.309050 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:35:03.317190 4589 raft_consensus.cc:1261] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 1 FOLLOWER]: Refusing update from remote peer 20967ca359d9461ab4da01d7130d4a52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:03.317248 4353 raft_consensus.cc:1261] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f [term 1 FOLLOWER]: Refusing update from remote peer 20967ca359d9461ab4da01d7130d4a52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:03.317727 4769 consensus_queue.cc:1035] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d0b2ce59dcd444c8b38fdf2cb8b24c4" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:03.317971 4734 consensus_queue.cc:1035] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "152a093c665a4fc6a7c6b347504ad34f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:03.345058 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.339198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:43635 (local address 127.1.224.65:41497)
0809 03:35:03.339332 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:03.339337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:03.340833 (+ 1496us) server_negotiation.cc:407] Connection header received
0809 03:35:03.340963 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:03.340966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:03.341016 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:03.341072 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:03.341418 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.342327 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.343783 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.344124 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.344209 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.344594 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:03.344684 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:03.344740 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:03.344875 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 03:35:03.344925 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:35:03.345122 4819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.338778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:41497 (local address 127.1.224.66:43635)
0809 03:35:03.340731 (+ 1953us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:03.340748 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:03.340866 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:03.341110 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:03.341113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:03.341121 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:35:03.341266 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.341272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.342459 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.342462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.343523 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:03.343536 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.344241 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.344247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:03.344354 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.344421 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:35:03.344774 (+ 353us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:35:03.344782 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 03:35:03.344849 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 03:35:03.344950 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1812,"thread_start_us":124,"threads_started":1}
I20260809 03:35:03.370491 3812 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:03.370693 3812 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:03.373394 3812 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47bf3b076830404b9745cd50a6a440eb: Sending DeleteTablet for 3 replicas of tablet b268f988778048d08f474fe6366e519c
I20260809 03:35:03.374545 4557 tablet_service.cc:1514] Processing DeleteTablet for tablet b268f988778048d08f474fe6366e519c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:03 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:57224
I20260809 03:35:03.374785 4756 tablet_replica.cc:330] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: stopping tablet replica
I20260809 03:35:03.374931 4756 raft_consensus.cc:2229] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:03.374941 4066 tablet_service.cc:1514] Processing DeleteTablet for tablet b268f988778048d08f474fe6366e519c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:03 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:35116
I20260809 03:35:03.374961 4311 tablet_service.cc:1514] Processing DeleteTablet for tablet b268f988778048d08f474fe6366e519c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:03 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:55148
I20260809 03:35:03.375098 4756 raft_consensus.cc:2258] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:03.375147 4759 tablet_replica.cc:330] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52: stopping tablet replica
I20260809 03:35:03.375260 4759 raft_consensus.cc:2229] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:03.375592 4759 raft_consensus.cc:2258] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:03.375795 4757 tablet_replica.cc:330] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f: stopping tablet replica
I20260809 03:35:03.375924 4757 raft_consensus.cc:2229] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:03.376062 4757 raft_consensus.cc:2258] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:03.377193 4756 ts_tablet_manager.cc:1905] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:03.381349 4759 ts_tablet_manager.cc:1905] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:03.384732 4759 ts_tablet_manager.cc:1918] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:03.384841 4759 log.cc:1196] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-0/wal/wals/b268f988778048d08f474fe6366e519c
Aug 09 03:35:03 dist-test-slave-jq4t krb5kdc[3743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246502, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.224.65@KRBTEST.COM
I20260809 03:35:03.385154 4759 ts_tablet_manager.cc:1939] T b268f988778048d08f474fe6366e519c P 20967ca359d9461ab4da01d7130d4a52: Deleting consensus metadata
I20260809 03:35:03.386087 3789 catalog_manager.cc:4841] TS 20967ca359d9461ab4da01d7130d4a52 (127.1.224.65:41497): tablet b268f988778048d08f474fe6366e519c (table test-table [id=4f61259a40184f66aafb4126a1b070a1]) successfully deleted
I20260809 03:35:03.386929 4757 ts_tablet_manager.cc:1905] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:03.389950 4757 ts_tablet_manager.cc:1918] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:03.390048 4757 log.cc:1196] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/b268f988778048d08f474fe6366e519c
I20260809 03:35:03.390311 4757 ts_tablet_manager.cc:1939] T b268f988778048d08f474fe6366e519c P 152a093c665a4fc6a7c6b347504ad34f: Deleting consensus metadata
I20260809 03:35:03.391220 3789 catalog_manager.cc:4841] TS 152a093c665a4fc6a7c6b347504ad34f (127.1.224.66:33323): tablet b268f988778048d08f474fe6366e519c (table test-table [id=4f61259a40184f66aafb4126a1b070a1]) successfully deleted
I20260809 03:35:03.391309 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.376171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35136 (local address 127.1.224.65:41497)
0809 03:35:03.377321 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:03.377326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:03.377340 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:35:03.377379 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:03.377382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:03.377426 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:03.377510 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:03.378600 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.379449 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.380761 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.381114 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.381192 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.385918 (+ 4726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:03.385946 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:03.385963 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:03.385998 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:03.388620 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:03.389325 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:03.389332 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:03.389334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:03.389443 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:03.389792 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:03.389796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:03.389798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:03.390160 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:03.390383 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:03.390961 (+ 578us) server_negotiation.cc:299] Negotiation successful
0809 03:35:03.391111 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1068}
I20260809 03:35:03.391654 4756 ts_tablet_manager.cc:1918] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:03.391749 4756 log.cc:1196] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/b268f988778048d08f474fe6366e519c
I20260809 03:35:03.392036 4756 ts_tablet_manager.cc:1939] T b268f988778048d08f474fe6366e519c P 6d0b2ce59dcd444c8b38fdf2cb8b24c4: Deleting consensus metadata
I20260809 03:35:03.392894 3790 catalog_manager.cc:4841] TS 6d0b2ce59dcd444c8b38fdf2cb8b24c4 (127.1.224.67:43949): tablet b268f988778048d08f474fe6366e519c (table test-table [id=4f61259a40184f66aafb4126a1b070a1]) successfully deleted
W20260809 03:35:03.392928 4158 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35136
I20260809 03:35:03.404742 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:03.394929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37458 (local address 127.1.224.126:39521)
0809 03:35:03.395070 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:03.395075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:03.395211 (+ 136us) server_negotiation.cc:407] Connection header received
0809 03:35:03.395317 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:03.395321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:03.395373 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:03.395460 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:03.396351 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.397317 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.398474 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:03.398832 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:03.398936 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:03.400867 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:03.400872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:03.400875 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:03.400907 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:03.402822 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:03.403317 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:03.403321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:03.403323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:03.403418 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:03.403720 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:03.403723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:03.403725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:03.403968 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:03.404110 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:03.404387 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 03:35:03.404553 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 03:35:03.405221 3812 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37458
I20260809 03:35:03.406849 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 3943
I20260809 03:35:03.425592 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 4191
I20260809 03:35:03.441951 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 4441
I20260809 03:35:03.462206 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 3764
2026-08-09T03:35:03Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3114 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:03 dist-test-slave-jq4t krb5kdc[4832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:03 dist-test-slave-jq4t krb5kdc[4832](info): set up 2 sockets
Aug 09 03:35:03 dist-test-slave-jq4t krb5kdc[4832](info): commencing operation
krb5kdc: starting...
W20260809 03:35:04.719895 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:35:04 dist-test-slave-jq4t krb5kdc[4832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:35:04.853652 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:36931
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:36931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:04.973651 4857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:04.973974 4857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:04.974018 4857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:04.978253 4857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:04.978331 4857 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 03:35:04.978348 4857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:04.978366 4857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:04.978384 4857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:04.978399 4857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:04.984253 4857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:36931
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:36931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:04.985684 4857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:04.986858 4857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:04.996120 4867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:04.996120 4869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:04.996368 4857 server_base.cc:1034] running on GCE node
W20260809 03:35:04.996145 4871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:04.997113 4857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:04.998279 4857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:04.999467 4857 hybrid_clock.cc:648] HybridClock initialized: now 1786246504999432 us; error 50 us; skew 500 ppm
Aug 09 03:35:04 dist-test-slave-jq4t krb5kdc[4832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:05.002884 4857 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:35:05.003973 4857 webserver.cc:460] Webserver started at http://127.1.224.126:36323/ using document root <none> and password file <none>
I20260809 03:35:05.004693 4857 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:05.004765 4857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:05.005010 4857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:05.007279 4857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "c4ee89b59b9d4024b84718d215f77fc6"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-jq4t"
server_key: "d875e63a96e4f42c78bdd2aeb92f418d"
server_key_iv: "581a58ba2b3d99acfc30716922259146"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.007905 4857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "c4ee89b59b9d4024b84718d215f77fc6"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-jq4t"
server_key: "d875e63a96e4f42c78bdd2aeb92f418d"
server_key_iv: "581a58ba2b3d99acfc30716922259146"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.012109 4857 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 03:35:05.015053 4882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.016444 4857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:05.016672 4857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "c4ee89b59b9d4024b84718d215f77fc6"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-jq4t"
server_key: "d875e63a96e4f42c78bdd2aeb92f418d"
server_key_iv: "581a58ba2b3d99acfc30716922259146"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.016808 4857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:05.029121 4857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:05.032708 4857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:05.032927 4857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:05.041291 4857 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:36931
I20260809 03:35:05.041309 5027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:36931 every 8 connection(s)
I20260809 03:35:05.042419 4857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:05.045444 5028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:05.051288 5028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6: Bootstrap starting.
I20260809 03:35:05.051723 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 4857
I20260809 03:35:05.051851 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:05.052160 1921 external_mini_cluster.cc:1428] Setting key f25fcc10bccede065297f88493056ba7
I20260809 03:35:05.054324 5028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:05.055136 5028 log.cc:824] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:05.057426 5028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6: No bootstrap required, opened a new log
Aug 09 03:35:05 dist-test-slave-jq4t krb5kdc[4832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:05.062275 5028 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ee89b59b9d4024b84718d215f77fc6" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 36931 } }
I20260809 03:35:05.062539 5028 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:05.062633 5028 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ee89b59b9d4024b84718d215f77fc6, State: Initialized, Role: FOLLOWER
I20260809 03:35:05.063108 5028 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ee89b59b9d4024b84718d215f77fc6" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 36931 } }
I20260809 03:35:05.063268 5028 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:05.063355 5028 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:05.063479 5028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:05.064445 5028 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ee89b59b9d4024b84718d215f77fc6" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 36931 } }
I20260809 03:35:05.064886 5028 leader_election.cc:302] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4ee89b59b9d4024b84718d215f77fc6; no voters:
I20260809 03:35:05.065289 5028 leader_election.cc:288] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:05.065373 5038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:05.065569 5038 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [term 1 LEADER]: Becoming Leader. State: Replica: c4ee89b59b9d4024b84718d215f77fc6, State: Running, Role: LEADER
I20260809 03:35:05.065927 5038 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ee89b59b9d4024b84718d215f77fc6" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 36931 } }
I20260809 03:35:05.066721 5028 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:05.067641 5041 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4ee89b59b9d4024b84718d215f77fc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ee89b59b9d4024b84718d215f77fc6" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 36931 } } }
I20260809 03:35:05.067761 5041 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:05.068114 5040 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4ee89b59b9d4024b84718d215f77fc6. Latest consensus state: current_term: 1 leader_uuid: "c4ee89b59b9d4024b84718d215f77fc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ee89b59b9d4024b84718d215f77fc6" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 36931 } } }
I20260809 03:35:05.068192 5040 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:05.068188 5056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:05.068799 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.053688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40588 (local address 127.1.224.126:36931)
0809 03:35:05.054183 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.054201 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.054238 (+ 37us) server_negotiation.cc:407] Connection header received
0809 03:35:05.054978 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.054998 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.055393 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.055772 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.056660 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.057743 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.058912 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.059404 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.059526 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.062457 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.062462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.062476 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.062506 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.064550 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.065100 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.065107 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.065113 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.065242 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.065666 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.065670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.065673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.066275 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.067072 (+ 797us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.067240 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.067659 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":157,"threads_started":1}
I20260809 03:35:05.071795 5056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:05.077919 5056 catalog_manager.cc:1347] Generated new cluster ID: 2fe709171569407b94022352751a32a6
I20260809 03:35:05.078007 5056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:05.099774 5056 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:05.101234 5056 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:05.111697 5056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6: Generated new TSK 0
I20260809 03:35:05.112658 5056 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:35:05.152154 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:36931
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:05.272529 5080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:05.272873 5080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:05.272962 5080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:05.277163 5080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:05.277258 5080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:05.277335 5080 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 03:35:05.277380 5080 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:35:05.277447 5080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:05.282967 5080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.224.126:36931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:05.284189 5080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:05.285374 5080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:05.295485 5088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:05.295485 5089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:05.295657 5093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:05.295755 5080 server_base.cc:1034] running on GCE node
I20260809 03:35:05.296334 5080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:05.297048 5080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:05.298247 5080 hybrid_clock.cc:648] HybridClock initialized: now 1786246505298233 us; error 28 us; skew 500 ppm
Aug 09 03:35:05 dist-test-slave-jq4t krb5kdc[4832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:05.301189 5080 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:35:05.302119 5080 webserver.cc:460] Webserver started at http://127.1.224.65:34649/ using document root <none> and password file <none>
I20260809 03:35:05.302724 5080 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:05.302820 5080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:05.303076 5080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:05.305313 5080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "a3e45c808a6846548792c864a23f7082"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-jq4t"
server_key: "3f335408d45fba2226edfc97ebda78ac"
server_key_iv: "30f5f68b3ebd90dc83f127ee8a1e1b48"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.305894 5080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "a3e45c808a6846548792c864a23f7082"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-jq4t"
server_key: "3f335408d45fba2226edfc97ebda78ac"
server_key_iv: "30f5f68b3ebd90dc83f127ee8a1e1b48"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.309710 5080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:35:05.312158 5105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.313493 5080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:05.313632 5080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "a3e45c808a6846548792c864a23f7082"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-jq4t"
server_key: "3f335408d45fba2226edfc97ebda78ac"
server_key_iv: "30f5f68b3ebd90dc83f127ee8a1e1b48"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.313747 5080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:05.334455 5080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:05.338104 5080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:05.338342 5080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:05.340030 5080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:05.340116 5080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.340195 5080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:05.340234 5080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:35:05 dist-test-slave-jq4t krb5kdc[4832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:05.352366 5080 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:39897
I20260809 03:35:05.352448 5321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:39897 every 8 connection(s)
I20260809 03:35:05.353619 5080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:05.356195 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.341164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:48515 (local address 127.1.224.126:36931)
0809 03:35:05.341311 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.341315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.342455 (+ 1140us) server_negotiation.cc:407] Connection header received
0809 03:35:05.343644 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.343648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.343714 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.343843 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.345732 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.346405 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.348500 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.348948 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.349058 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.352496 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.352501 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.352504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.352537 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.354224 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.354779 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.354783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.354784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.354871 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.355186 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.355188 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.355189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.355387 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.355476 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.355952 (+ 476us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.356068 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:35:05.357124 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.341728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:36931 (local address 127.1.224.65:48515)
0809 03:35:05.342284 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.342331 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.343373 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.344100 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.344115 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.344684 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:05.345508 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.345536 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.346536 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.346540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.348284 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.348302 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.349147 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.349151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.349405 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.350262 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:05.350290 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:05.352252 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:05.354387 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.354394 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.354412 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.354697 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.354964 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.354966 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.354967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.355099 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.355491 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:05.355496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:05.355737 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.355897 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.356157 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":153,"threads_started":1}
I20260809 03:35:05.358553 5322 heartbeater.cc:347] Connected to a master server at 127.1.224.126:36931
I20260809 03:35:05.358796 5322 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:05.359325 5322 heartbeater.cc:511] Master 127.1.224.126:36931 requested a full tablet report, sending...
I20260809 03:35:05.360730 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 5080
I20260809 03:35:05.360867 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:05.361057 4924 ts_manager.cc:194] Registered new tserver with Master: a3e45c808a6846548792c864a23f7082 (127.1.224.65:39897)
I20260809 03:35:05.361255 1921 external_mini_cluster.cc:1428] Setting key 15197e22fe7590080cc7d6bdc1f05286
I20260809 03:35:05.362857 4924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:48515
I20260809 03:35:05.369626 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.361294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:57015 (local address 127.1.224.126:36931)
0809 03:35:05.361497 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.361501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.361845 (+ 344us) server_negotiation.cc:407] Connection header received
0809 03:35:05.362016 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.362019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.362074 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.362161 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.362619 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.363229 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.364337 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.364763 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.364891 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.366112 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.366116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.366119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.366144 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.367801 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.368272 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.368276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.368277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.368368 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.368730 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.368732 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.368734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.368921 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.369000 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.369357 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.369491 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":120}
I20260809 03:35:05.369791 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.361201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:36931 (local address 127.1.224.65:57015)
0809 03:35:05.361746 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.361759 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.361925 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.362180 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.362183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.362395 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:05.362502 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.362507 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.363384 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.363389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.364204 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.364215 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.364897 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.364900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.364990 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.365594 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:05.365610 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:05.365970 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:05.367940 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.367945 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.367947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.368162 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.368462 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.368466 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.368468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.368615 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.369033 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:05.369038 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:05.369166 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.369300 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.369507 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:35:05.396885 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:36931
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:05.521951 5336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:05.522259 5336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:05.522310 5336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:05.526443 5336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:05.526517 5336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:05.526559 5336 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 03:35:05.526573 5336 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:35:05.526607 5336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:05.532205 5336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.224.126:36931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:05.533476 5336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:05.534560 5336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:35:05.544766 5336 server_base.cc:1034] running on GCE node
W20260809 03:35:05.544726 5351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:05.544726 5348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:05.544751 5349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:05.545544 5336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:05.546190 5336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:05.547358 5336 hybrid_clock.cc:648] HybridClock initialized: now 1786246505547330 us; error 39 us; skew 500 ppm
Aug 09 03:35:05 dist-test-slave-jq4t krb5kdc[4832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:05.550724 5336 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:35:05.551783 5336 webserver.cc:460] Webserver started at http://127.1.224.66:33695/ using document root <none> and password file <none>
I20260809 03:35:05.552431 5336 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:05.552559 5336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:05.552822 5336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:05.555114 5336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "eb967b49e75e4747912973867608459d"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-jq4t"
server_key: "43a14cb9606afa81fd458b3241fe26b9"
server_key_iv: "ca510e5ec9713ddd4aa94c4f8c213080"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.555749 5336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "eb967b49e75e4747912973867608459d"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-jq4t"
server_key: "43a14cb9606afa81fd458b3241fe26b9"
server_key_iv: "ca510e5ec9713ddd4aa94c4f8c213080"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.559940 5336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 03:35:05.562547 5360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.563817 5336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:05.563990 5336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "eb967b49e75e4747912973867608459d"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-jq4t"
server_key: "43a14cb9606afa81fd458b3241fe26b9"
server_key_iv: "ca510e5ec9713ddd4aa94c4f8c213080"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.564114 5336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:05.610484 5336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:05.614545 5336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:05.614784 5336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:05.616418 5336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:05.616472 5336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.616626 5336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:05.616683 5336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:35:05 dist-test-slave-jq4t krb5kdc[4832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:05.628180 5336 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:34829
I20260809 03:35:05.628219 5578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:34829 every 8 connection(s)
I20260809 03:35:05.629357 5336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:05.632469 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.617393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:47351 (local address 127.1.224.126:36931)
0809 03:35:05.617544 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.617549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.618563 (+ 1014us) server_negotiation.cc:407] Connection header received
0809 03:35:05.619671 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.619676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.619746 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.619811 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.621872 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.622814 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.624882 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.625115 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.625207 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.628728 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.628732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.628734 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.628773 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.630538 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.631079 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.631082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.631084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.631170 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.631455 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.631457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.631458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.631652 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.631726 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.632225 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.632333 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 03:35:05.633389 5379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.617904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:36931 (local address 127.1.224.66:47351)
0809 03:35:05.618393 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.618447 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.619358 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.620319 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.620328 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.620923 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:05.621663 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.621687 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.622984 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.622989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.624726 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.624744 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.625311 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.625317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.625556 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.626398 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:05.626425 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:05.628522 (+ 2097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:05.630668 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.630674 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.630685 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.630964 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.631231 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.631233 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.631234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.631364 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.631742 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:05.631747 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:05.631992 (+ 245us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.632158 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.632424 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":124,"threads_started":1}
I20260809 03:35:05.634598 5579 heartbeater.cc:347] Connected to a master server at 127.1.224.126:36931
I20260809 03:35:05.634748 5579 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:05.635360 5579 heartbeater.cc:511] Master 127.1.224.126:36931 requested a full tablet report, sending...
I20260809 03:35:05.636628 4922 ts_manager.cc:194] Registered new tserver with Master: eb967b49e75e4747912973867608459d (127.1.224.66:34829)
I20260809 03:35:05.637456 4922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:47351
I20260809 03:35:05.637661 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 5336
I20260809 03:35:05.637784 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:05.638132 1921 external_mini_cluster.cc:1428] Setting key 698b66934a40d0abd76fa1186bd40c93
I20260809 03:35:05.645356 5588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.637113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:36931 (local address 127.1.224.66:33683)
0809 03:35:05.637501 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.637513 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.637676 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.638075 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.638077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.638324 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:05.638436 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.638444 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.639462 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.639466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.640061 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.640071 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.640568 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.640572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.640674 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.641249 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:05.641260 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:05.641730 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:05.643559 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.643566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.643571 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.643854 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.644132 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.644138 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.644142 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.644255 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.644778 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:05.644783 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:05.644888 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.645002 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.645170 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":104,"threads_started":1}
I20260809 03:35:05.645345 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.637192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:33683 (local address 127.1.224.126:36931)
0809 03:35:05.637337 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.637342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.637600 (+ 258us) server_negotiation.cc:407] Connection header received
0809 03:35:05.637808 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.637813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.637861 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.637923 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.638571 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.639311 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.640195 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.640419 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.640563 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.641880 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.641883 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.641885 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.641908 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.643444 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.643955 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.643958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.643959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.644037 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.644354 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.644357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.644358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.644656 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.644774 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.645063 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.645176 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:35:05.669826 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:36931
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:05.791517 5596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:05.791795 5596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:05.791849 5596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:05.795964 5596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:05.796023 5596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:05.796062 5596 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 03:35:05.796087 5596 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:35:05.796123 5596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:05.801431 5596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.224.126:36931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:05.802481 5596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:05.803493 5596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:05.814311 5610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:05.814291 5609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:05.814291 5612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:05.814355 5596 server_base.cc:1034] running on GCE node
I20260809 03:35:05.815030 5596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:05.815773 5596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:05.816963 5596 hybrid_clock.cc:648] HybridClock initialized: now 1786246505816928 us; error 55 us; skew 500 ppm
Aug 09 03:35:05 dist-test-slave-jq4t krb5kdc[4832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:05.820115 5596 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:35:05.821237 5596 webserver.cc:460] Webserver started at http://127.1.224.67:33079/ using document root <none> and password file <none>
I20260809 03:35:05.821875 5596 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:05.821973 5596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:05.822227 5596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:05.824532 5596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "c747975b209c4f20ad38ed36e3f4a3a0"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-jq4t"
server_key: "4384b7cde02c2fda84d686c99b62cc34"
server_key_iv: "faf1613a82b32aae3937edab26b51ffa"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.825153 5596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "c747975b209c4f20ad38ed36e3f4a3a0"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-jq4t"
server_key: "4384b7cde02c2fda84d686c99b62cc34"
server_key_iv: "faf1613a82b32aae3937edab26b51ffa"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.829130 5596 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:35:05.831634 5623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.832943 5596 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:05.833097 5596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "c747975b209c4f20ad38ed36e3f4a3a0"
format_stamp: "Formatted at 2026-08-09 03:35:05 on dist-test-slave-jq4t"
server_key: "4384b7cde02c2fda84d686c99b62cc34"
server_key_iv: "faf1613a82b32aae3937edab26b51ffa"
server_key_version: "encryptionkey@0"
I20260809 03:35:05.833213 5596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:05.849083 5596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:05.852629 5596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:05.852870 5596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:05.854616 5596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:05.854696 5596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:05.854766 5596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:05.854820 5596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:35:05 dist-test-slave-jq4t krb5kdc[4832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:05.866557 5596 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:37729
I20260809 03:35:05.866696 5820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:37729 every 8 connection(s)
I20260809 03:35:05.867697 5596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:05.867920 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 5596
I20260809 03:35:05.868074 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:05.868419 1921 external_mini_cluster.cc:1428] Setting key 69ae9de7ca0605f0aefcace3b148e61e
I20260809 03:35:05.869583 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.855642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:35539 (local address 127.1.224.126:36931)
0809 03:35:05.855784 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.855788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.856950 (+ 1162us) server_negotiation.cc:407] Connection header received
0809 03:35:05.858192 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.858198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.858274 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.858386 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.859906 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.860684 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.862039 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.862323 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.862445 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.865223 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.865234 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.865237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.865263 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.866974 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.867506 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.867510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.867512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.867627 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.868103 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.868106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.868108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.868364 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.868513 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.869275 (+ 762us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.869413 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:35:05.870478 5633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.856231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:36931 (local address 127.1.224.67:35539)
0809 03:35:05.856767 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.856811 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.857936 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.858582 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.858594 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.859112 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:05.859725 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.859741 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.860813 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.860817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.861860 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.861871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.862503 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.862508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.862718 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.863387 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:05.863422 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:05.865043 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:05.867104 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.867110 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.867125 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.867392 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.867779 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.867784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.867786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.867971 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.868653 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:05.868660 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:05.869006 (+ 346us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.869223 (+ 217us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.869563 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":171,"threads_started":1}
I20260809 03:35:05.871704 5823 heartbeater.cc:347] Connected to a master server at 127.1.224.126:36931
I20260809 03:35:05.871869 5823 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:05.872395 5823 heartbeater.cc:511] Master 127.1.224.126:36931 requested a full tablet report, sending...
I20260809 03:35:05.873538 4924 ts_manager.cc:194] Registered new tserver with Master: c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729)
I20260809 03:35:05.874348 4924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:35539
I20260809 03:35:05.880785 5839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.874029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:36931 (local address 127.1.224.67:53243)
0809 03:35:05.874326 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.874338 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.874451 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.874725 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.874728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.874957 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:05.875118 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.875124 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.876112 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.876114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.876756 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.876764 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.877172 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.877175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.877249 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.877696 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:05.877707 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:05.878035 (+ 328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:05.879486 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.879489 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.879490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.879678 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.879915 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:05.879917 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:05.879919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:05.880007 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:05.880317 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:05.880319 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:05.880392 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.880518 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.880641 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260809 03:35:05.880790 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.874108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:53243 (local address 127.1.224.126:36931)
0809 03:35:05.874236 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.874240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.874416 (+ 176us) server_negotiation.cc:407] Connection header received
0809 03:35:05.874536 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.874539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.874605 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.874681 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.875238 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.875981 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.876860 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.877072 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.877153 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.878157 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.878160 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.878162 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.878183 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.879385 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.879761 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.879764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.879765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.879836 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.880085 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.880088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.880089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.880238 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.880311 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.880586 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.880687 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:35:05.882848 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:35:05 dist-test-slave-jq4t krb5kdc[4832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:35:05 dist-test-slave-jq4t krb5kdc[4832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:05.910856 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.902055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40482 (local address 127.1.224.126:36931)
0809 03:35:05.902197 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.902201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.902213 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:35:05.902259 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.902261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.902303 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.902369 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:05.903129 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.903671 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.904562 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.904779 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.904861 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.907529 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:05.907533 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:05.907535 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:05.907562 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:05.908996 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.909493 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.909501 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.909503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.909589 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:05.909894 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:05.909897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:05.909899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:05.910103 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:05.910245 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.910514 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.910643 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:35:05.914227 4922 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:05.917483 4922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:05.932436 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.926318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51520 (local address 127.1.224.66:34829)
0809 03:35:05.926780 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.926795 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.926823 (+ 28us) server_negotiation.cc:407] Connection header received
0809 03:35:05.927327 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.927333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.927573 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.927742 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:05.928075 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.929355 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.930923 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.931722 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.931843 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.932043 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.932176 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.932279 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":126,"threads_started":1}
I20260809 03:35:05.932426 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.926154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:34829 (local address 127.0.0.1:51520)
0809 03:35:05.926528 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.926576 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.927091 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.927827 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.927832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.927849 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:05.927961 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.927965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.929492 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.929495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.930787 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.930797 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.931854 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.931858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.931967 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.932008 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.932129 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.932253 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":61,"thread_start_us":114,"threads_started":1}
I20260809 03:35:05.933221 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.926354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:37729 (local address 127.0.0.1:58394)
0809 03:35:05.927589 (+ 1235us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.927605 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.927702 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.928345 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.928348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.928362 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:05.928470 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.928498 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.929822 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.929825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.931078 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.931089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.932364 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.932368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.932500 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.932942 (+ 442us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.933018 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.933089 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1174,"mutex_wait_us":67,"thread_start_us":99,"threads_started":1}
I20260809 03:35:05.933517 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.926920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39897 (local address 127.0.0.1:42438)
0809 03:35:05.928129 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:05.928139 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:05.928237 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:05.928713 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:05.928716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:05.928734 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:05.928810 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.928815 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.930467 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.930471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.931481 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:05.931489 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.932778 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.932782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:05.932906 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.933195 (+ 289us) client_negotiation.cc:769] Sending connection context
0809 03:35:05.933266 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 03:35:05.933362 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1161,"mutex_wait_us":255,"thread_start_us":108,"threads_started":1}
I20260809 03:35:05.933668 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.926965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.1.224.67:37729)
0809 03:35:05.927820 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.927833 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.927857 (+ 24us) server_negotiation.cc:407] Connection header received
0809 03:35:05.927937 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.927941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.928142 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.928294 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:05.928620 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.929691 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.931214 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.932001 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.932141 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.932298 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.933408 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.933525 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":139,"threads_started":1}
I20260809 03:35:05.933974 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:05.927371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42438 (local address 127.1.224.65:39897)
0809 03:35:05.927699 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:05.927705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:05.928229 (+ 524us) server_negotiation.cc:407] Connection header received
0809 03:35:05.928345 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:05.928350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:05.928572 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:05.928729 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:05.928924 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.930241 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.931607 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:05.932402 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:05.932960 (+ 558us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:05.933134 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:05.933734 (+ 600us) server_negotiation.cc:299] Negotiation successful
0809 03:35:05.933854 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":104,"threads_started":1}
I20260809 03:35:05.936261 5695 tablet_service.cc:1467] Processing CreateTablet for tablet de07d26cbbec4139a9e8652cda09cef8 (DEFAULT_TABLE table=test-table [id=8953746d04d14f7d9f0f152d3bcc4186]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:05.936261 5460 tablet_service.cc:1467] Processing CreateTablet for tablet de07d26cbbec4139a9e8652cda09cef8 (DEFAULT_TABLE table=test-table [id=8953746d04d14f7d9f0f152d3bcc4186]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:05.936421 5196 tablet_service.cc:1467] Processing CreateTablet for tablet de07d26cbbec4139a9e8652cda09cef8 (DEFAULT_TABLE table=test-table [id=8953746d04d14f7d9f0f152d3bcc4186]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:05.937549 5695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de07d26cbbec4139a9e8652cda09cef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:05.937546 5460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de07d26cbbec4139a9e8652cda09cef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:05.937580 5196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de07d26cbbec4139a9e8652cda09cef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:05.944314 5869 tablet_bootstrap.cc:492] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082: Bootstrap starting.
I20260809 03:35:05.946799 5869 tablet_bootstrap.cc:654] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:05.947194 5872 tablet_bootstrap.cc:492] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d: Bootstrap starting.
I20260809 03:35:05.947773 5869 log.cc:824] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:05.948052 5870 tablet_bootstrap.cc:492] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0: Bootstrap starting.
I20260809 03:35:05.949596 5872 tablet_bootstrap.cc:654] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:05.950292 5869 tablet_bootstrap.cc:492] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082: No bootstrap required, opened a new log
I20260809 03:35:05.950488 5872 log.cc:824] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:05.950511 5869 ts_tablet_manager.cc:1397] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:35:05.950923 5870 tablet_bootstrap.cc:654] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:05.951767 5870 log.cc:824] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:05.952404 5872 tablet_bootstrap.cc:492] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d: No bootstrap required, opened a new log
I20260809 03:35:05.952652 5872 ts_tablet_manager.cc:1397] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:35:05.954098 5870 tablet_bootstrap.cc:492] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0: No bootstrap required, opened a new log
I20260809 03:35:05.954301 5870 ts_tablet_manager.cc:1397] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:35:05.955484 5869 raft_consensus.cc:348] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:05.955788 5869 raft_consensus.cc:374] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:05.955849 5869 raft_consensus.cc:729] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3e45c808a6846548792c864a23f7082, State: Initialized, Role: FOLLOWER
I20260809 03:35:05.955847 5872 raft_consensus.cc:348] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:05.956099 5872 raft_consensus.cc:374] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:05.956158 5872 raft_consensus.cc:729] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb967b49e75e4747912973867608459d, State: Initialized, Role: FOLLOWER
I20260809 03:35:05.956377 5869 consensus_queue.cc:260] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:05.956638 5872 consensus_queue.cc:260] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:05.957193 5322 heartbeater.cc:503] Master 127.1.224.126:36931 was elected leader, sending a full tablet report...
I20260809 03:35:05.957443 5869 ts_tablet_manager.cc:1428] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 03:35:05.957631 5872 ts_tablet_manager.cc:1428] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:35:05.957985 5579 heartbeater.cc:503] Master 127.1.224.126:36931 was elected leader, sending a full tablet report...
I20260809 03:35:05.958182 5870 raft_consensus.cc:348] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:05.958436 5870 raft_consensus.cc:374] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:05.958505 5870 raft_consensus.cc:729] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c747975b209c4f20ad38ed36e3f4a3a0, State: Initialized, Role: FOLLOWER
I20260809 03:35:05.958917 5870 consensus_queue.cc:260] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:05.959630 5823 heartbeater.cc:503] Master 127.1.224.126:36931 was elected leader, sending a full tablet report...
I20260809 03:35:05.959964 5870 ts_tablet_manager.cc:1428] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260809 03:35:06.105299 5324 tablet.cc:2369] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:35:06.119340 5825 tablet.cc:2369] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:35:06.130846 5580 tablet.cc:2369] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:06.230466 5880 raft_consensus.cc:482] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:06.230863 5880 raft_consensus.cc:504] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.232232 5880 leader_election.cc:288] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb967b49e75e4747912973867608459d (127.1.224.66:34829), c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729)
I20260809 03:35:06.237371 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.232736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:52197 (local address 127.1.224.67:37729)
0809 03:35:06.232898 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.232904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.233160 (+ 256us) server_negotiation.cc:407] Connection header received
0809 03:35:06.233260 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.233263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.233318 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.233417 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:06.233926 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.234759 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.236199 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.236713 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.236875 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.236958 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.237171 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.237246 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:35:06.237362 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.232778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:42973 (local address 127.1.224.66:34829)
0809 03:35:06.232904 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.232908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.233099 (+ 191us) server_negotiation.cc:407] Connection header received
0809 03:35:06.233261 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.233264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.233318 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.233417 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:06.233926 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.234757 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.236231 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.236711 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.236879 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.236939 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.237131 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.237219 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:35:06.237408 5886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.232541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:37729 (local address 127.1.224.65:52197)
0809 03:35:06.233060 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.233074 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.233159 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.233449 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.233452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.233500 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:06.233756 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.233766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.234891 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.234895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.236036 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.236049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.236897 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.236900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.237021 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.237040 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.237140 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.237211 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":285,"spinlock_wait_cycles":53504,"thread_start_us":100,"threads_started":1}
I20260809 03:35:06.237443 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.232541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:34829 (local address 127.1.224.65:42973)
0809 03:35:06.232980 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.232996 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.233151 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.233449 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.233452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.233499 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:06.233756 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.233764 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.234891 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.234895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.236038 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.236052 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.236863 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.236867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.236967 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.237001 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.237083 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.237185 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":236,"spinlock_wait_cycles":38656,"thread_start_us":121,"threads_started":1}
I20260809 03:35:06.238116 5713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de07d26cbbec4139a9e8652cda09cef8" candidate_uuid: "a3e45c808a6846548792c864a23f7082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" is_pre_election: true
I20260809 03:35:06.238212 5493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de07d26cbbec4139a9e8652cda09cef8" candidate_uuid: "a3e45c808a6846548792c864a23f7082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb967b49e75e4747912973867608459d" is_pre_election: true
I20260809 03:35:06.238487 5713 raft_consensus.cc:2454] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3e45c808a6846548792c864a23f7082 in term 0.
I20260809 03:35:06.238531 5493 raft_consensus.cc:2454] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3e45c808a6846548792c864a23f7082 in term 0.
I20260809 03:35:06.239050 5107 leader_election.cc:302] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a3e45c808a6846548792c864a23f7082, c747975b209c4f20ad38ed36e3f4a3a0, eb967b49e75e4747912973867608459d; no voters:
I20260809 03:35:06.239367 5880 raft_consensus.cc:2781] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:06.239452 5880 raft_consensus.cc:482] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:06.239485 5880 raft_consensus.cc:3037] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.240621 5880 raft_consensus.cc:504] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.241036 5880 leader_election.cc:288] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [CANDIDATE]: Term 1 election: Requested vote from peers eb967b49e75e4747912973867608459d (127.1.224.66:34829), c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729)
I20260809 03:35:06.241376 5493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de07d26cbbec4139a9e8652cda09cef8" candidate_uuid: "a3e45c808a6846548792c864a23f7082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb967b49e75e4747912973867608459d"
I20260809 03:35:06.241416 5713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de07d26cbbec4139a9e8652cda09cef8" candidate_uuid: "a3e45c808a6846548792c864a23f7082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c747975b209c4f20ad38ed36e3f4a3a0"
I20260809 03:35:06.241525 5493 raft_consensus.cc:3037] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.241535 5713 raft_consensus.cc:3037] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.242580 5493 raft_consensus.cc:2454] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3e45c808a6846548792c864a23f7082 in term 1.
I20260809 03:35:06.242937 5119 leader_election.cc:302] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3e45c808a6846548792c864a23f7082, eb967b49e75e4747912973867608459d; no voters:
I20260809 03:35:06.243009 5713 raft_consensus.cc:2454] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3e45c808a6846548792c864a23f7082 in term 1.
I20260809 03:35:06.243136 5880 raft_consensus.cc:2781] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:06.243376 5880 raft_consensus.cc:686] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 1 LEADER]: Becoming Leader. State: Replica: a3e45c808a6846548792c864a23f7082, State: Running, Role: LEADER
I20260809 03:35:06.243753 5880 consensus_queue.cc:237] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.247066 4924 catalog_manager.cc:5495] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3e45c808a6846548792c864a23f7082 (127.1.224.65). New cstate: current_term: 1 leader_uuid: "a3e45c808a6846548792c864a23f7082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } health_report { overall_health: UNKNOWN } } }
Aug 09 03:35:06 dist-test-slave-jq4t krb5kdc[4832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246504, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:06.328050 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.318710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:36931 (local address 127.0.0.1:40486)
0809 03:35:06.319056 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.319070 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.319183 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.319440 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.319442 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.319864 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:06.320024 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.320035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.321060 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.321070 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.321672 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.321681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.322099 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.322102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.322174 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.322844 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:06.322882 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:06.324757 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:06.326609 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:06.326618 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:06.326624 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:06.326855 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:06.327116 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:06.327118 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:06.327120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:06.327200 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:06.327558 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:06.327564 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:06.327673 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.327768 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.327895 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":161,"threads_started":1}
I20260809 03:35:06.328079 5034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.318794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40486 (local address 127.1.224.126:36931)
0809 03:35:06.318973 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.318978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.319158 (+ 180us) server_negotiation.cc:407] Connection header received
0809 03:35:06.319270 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.319273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.319319 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.319391 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:06.320149 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.320940 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.321777 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.321984 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.322075 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.324896 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:06.324899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:06.324901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:06.324923 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:06.326510 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:06.326955 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:06.326957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:06.326959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:06.327034 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:06.327281 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:06.327283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:06.327284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:06.327463 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:06.327539 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.327827 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.327966 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":85}
I20260809 03:35:06.332444 4924 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:40486:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 03:35:06.333308 4924 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:06.338763 5196 tablet_service.cc:1467] Processing CreateTablet for tablet 4e566aadb1a3440fad2a06660bf8b237 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9d468cc544f94befb28fb421cb193041]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:35:06.338842 5460 tablet_service.cc:1467] Processing CreateTablet for tablet 4e566aadb1a3440fad2a06660bf8b237 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9d468cc544f94befb28fb421cb193041]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:35:06.339578 5460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e566aadb1a3440fad2a06660bf8b237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:06.339589 5196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e566aadb1a3440fad2a06660bf8b237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:06.340997 5695 tablet_service.cc:1467] Processing CreateTablet for tablet 4e566aadb1a3440fad2a06660bf8b237 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9d468cc544f94befb28fb421cb193041]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:35:06.341730 5695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e566aadb1a3440fad2a06660bf8b237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:06.342540 5872 tablet_bootstrap.cc:492] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: Bootstrap starting.
I20260809 03:35:06.342901 5869 tablet_bootstrap.cc:492] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082: Bootstrap starting.
I20260809 03:35:06.343426 5872 tablet_bootstrap.cc:654] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:06.344451 5869 tablet_bootstrap.cc:654] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:06.344643 5872 tablet_bootstrap.cc:492] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: No bootstrap required, opened a new log
I20260809 03:35:06.344709 5872 ts_tablet_manager.cc:1397] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:35:06.344973 5870 tablet_bootstrap.cc:492] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0: Bootstrap starting.
I20260809 03:35:06.345181 5872 raft_consensus.cc:348] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.345288 5872 raft_consensus.cc:374] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:06.345321 5872 raft_consensus.cc:729] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb967b49e75e4747912973867608459d, State: Initialized, Role: FOLLOWER
I20260809 03:35:06.345487 5872 consensus_queue.cc:260] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.345747 5872 ts_tablet_manager.cc:1428] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:06.345907 5869 tablet_bootstrap.cc:492] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082: No bootstrap required, opened a new log
I20260809 03:35:06.345969 5869 ts_tablet_manager.cc:1397] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:06.346015 5870 tablet_bootstrap.cc:654] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:06.346458 5869 raft_consensus.cc:348] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.346588 5869 raft_consensus.cc:374] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:06.346624 5869 raft_consensus.cc:729] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3e45c808a6846548792c864a23f7082, State: Initialized, Role: FOLLOWER
I20260809 03:35:06.346731 5881 tablet_replica.cc:439] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } } }
I20260809 03:35:06.346752 5869 consensus_queue.cc:260] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.346884 5881 tablet_replica.cc:442] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:06.347028 5869 ts_tablet_manager.cc:1428] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:06.347262 5880 tablet_replica.cc:439] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } } }
I20260809 03:35:06.347399 5880 tablet_replica.cc:442] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:06.347435 5870 tablet_bootstrap.cc:492] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0: No bootstrap required, opened a new log
I20260809 03:35:06.347520 5870 ts_tablet_manager.cc:1397] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:06.348017 5870 raft_consensus.cc:348] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.348150 5870 raft_consensus.cc:374] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:06.348192 5870 raft_consensus.cc:729] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c747975b209c4f20ad38ed36e3f4a3a0, State: Initialized, Role: FOLLOWER
I20260809 03:35:06.348330 5870 consensus_queue.cc:260] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.348672 5870 ts_tablet_manager.cc:1428] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 03:35:06.348627 5882 tablet_replica.cc:439] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } } }
I20260809 03:35:06.348757 5882 tablet_replica.cc:442] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:06.394717 5881 raft_consensus.cc:482] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:06.394886 5881 raft_consensus.cc:504] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.396131 5881 leader_election.cc:288] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3e45c808a6846548792c864a23f7082 (127.1.224.65:39897), c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729)
I20260809 03:35:06.401317 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.396501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:37093 (local address 127.1.224.65:39897)
0809 03:35:06.396647 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.396652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.396772 (+ 120us) server_negotiation.cc:407] Connection header received
0809 03:35:06.397026 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.397030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.397081 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.397159 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:06.397640 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.398341 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.399962 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.400658 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.400801 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.400911 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.401096 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.401178 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:35:06.401340 5904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.396318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39897 (local address 127.1.224.66:37093)
0809 03:35:06.396680 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.396696 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.396856 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.397191 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.397195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.397241 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:06.397486 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.397496 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.398473 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.398475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.399812 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.399823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.400846 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.400851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.400953 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.400976 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.401038 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.401134 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":47,"spinlock_wait_cycles":7808,"thread_start_us":109,"threads_started":1}
I20260809 03:35:06.401479 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.396769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:53245 (local address 127.1.224.67:37729)
0809 03:35:06.396906 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.396910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.396949 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:35:06.397012 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.397015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.397064 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.397158 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:06.398022 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.398690 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.400038 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.400658 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.400801 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.400883 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.401271 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.401355 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:35:06.401872 5905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.396358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:37729 (local address 127.1.224.66:53245)
0809 03:35:06.396840 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.396853 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.396927 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.397191 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.397195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.397239 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:06.397878 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.397885 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.398812 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.398815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.399908 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.399917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.400974 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.400979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.401123 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.401148 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.401221 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.401290 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":70,"spinlock_wait_cycles":34944,"thread_start_us":105,"threads_started":1}
I20260809 03:35:06.402112 5232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e566aadb1a3440fad2a06660bf8b237" candidate_uuid: "eb967b49e75e4747912973867608459d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3e45c808a6846548792c864a23f7082" is_pre_election: true
I20260809 03:35:06.402426 5232 raft_consensus.cc:2454] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb967b49e75e4747912973867608459d in term 0.
I20260809 03:35:06.402457 5713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e566aadb1a3440fad2a06660bf8b237" candidate_uuid: "eb967b49e75e4747912973867608459d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" is_pre_election: true
I20260809 03:35:06.402604 5713 raft_consensus.cc:2454] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb967b49e75e4747912973867608459d in term 0.
I20260809 03:35:06.402951 5364 leader_election.cc:302] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3e45c808a6846548792c864a23f7082, eb967b49e75e4747912973867608459d; no voters:
I20260809 03:35:06.403267 5881 raft_consensus.cc:2781] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:06.403335 5881 raft_consensus.cc:482] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:06.403385 5881 raft_consensus.cc:3037] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.404414 5881 raft_consensus.cc:504] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.404848 5881 leader_election.cc:288] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [CANDIDATE]: Term 1 election: Requested vote from peers a3e45c808a6846548792c864a23f7082 (127.1.224.65:39897), c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729)
I20260809 03:35:06.405186 5232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e566aadb1a3440fad2a06660bf8b237" candidate_uuid: "eb967b49e75e4747912973867608459d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3e45c808a6846548792c864a23f7082"
I20260809 03:35:06.405243 5713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e566aadb1a3440fad2a06660bf8b237" candidate_uuid: "eb967b49e75e4747912973867608459d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c747975b209c4f20ad38ed36e3f4a3a0"
I20260809 03:35:06.405305 5232 raft_consensus.cc:3037] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.405395 5713 raft_consensus.cc:3037] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.406289 5232 raft_consensus.cc:2454] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb967b49e75e4747912973867608459d in term 1.
I20260809 03:35:06.406483 5713 raft_consensus.cc:2454] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb967b49e75e4747912973867608459d in term 1.
I20260809 03:35:06.406574 5364 leader_election.cc:302] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3e45c808a6846548792c864a23f7082, eb967b49e75e4747912973867608459d; no voters:
I20260809 03:35:06.406754 5881 raft_consensus.cc:2781] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:06.406996 5881 raft_consensus.cc:686] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [term 1 LEADER]: Becoming Leader. State: Replica: eb967b49e75e4747912973867608459d, State: Running, Role: LEADER
I20260809 03:35:06.407390 5881 consensus_queue.cc:237] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.410512 5907 tablet_replica.cc:439] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: TxnStatusTablet state changed. Reason: New leader eb967b49e75e4747912973867608459d. Latest consensus state: current_term: 1 leader_uuid: "eb967b49e75e4747912973867608459d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } } }
I20260809 03:35:06.410629 5907 tablet_replica.cc:442] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:35:06.410976 5909 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 03:35:06.411231 5909 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 03:35:06.411080 4924 catalog_manager.cc:5495] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d reported cstate change: term changed from 0 to 1, leader changed from <none> to eb967b49e75e4747912973867608459d (127.1.224.66). New cstate: current_term: 1 leader_uuid: "eb967b49e75e4747912973867608459d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } health_report { overall_health: UNKNOWN } } }
I20260809 03:35:06.436538 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.431878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:34829 (local address 127.0.0.1:51534)
0809 03:35:06.432093 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.432108 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.432209 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.432628 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.432630 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.432637 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:35:06.432780 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.432789 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.433846 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.433849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.434571 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.434601 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.435187 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.435189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.435258 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.435312 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.436182 (+ 870us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.436199 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.436260 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.436345 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 03:35:06.436534 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.431994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.1.224.66:34829)
0809 03:35:06.432156 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.432161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.432178 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:35:06.432371 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.432374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.432448 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.432591 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:06.432906 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.433742 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.434720 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.435077 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.435181 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.435464 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.436081 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.436203 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.436291 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.436352 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 03:35:06.442903 5713 raft_consensus.cc:1261] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Refusing update from remote peer eb967b49e75e4747912973867608459d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:06.442914 5232 raft_consensus.cc:1261] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Refusing update from remote peer eb967b49e75e4747912973867608459d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:06.443609 5907 consensus_queue.cc:1035] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:06.443843 5881 consensus_queue.cc:1035] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:06.447280 5880 tablet_replica.cc:439] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082: TxnStatusTablet state changed. Reason: New leader eb967b49e75e4747912973867608459d. Latest consensus state: current_term: 1 leader_uuid: "eb967b49e75e4747912973867608459d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } } }
I20260809 03:35:06.447422 5880 tablet_replica.cc:442] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:06.447485 5882 tablet_replica.cc:439] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0: TxnStatusTablet state changed. Reason: New leader eb967b49e75e4747912973867608459d. Latest consensus state: current_term: 1 leader_uuid: "eb967b49e75e4747912973867608459d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } } }
I20260809 03:35:06.447602 5882 tablet_replica.cc:442] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:06.451622 5881 tablet_replica.cc:439] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb967b49e75e4747912973867608459d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } } }
I20260809 03:35:06.451737 5881 tablet_replica.cc:442] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:35:06.453424 5882 tablet_replica.cc:439] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb967b49e75e4747912973867608459d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } } }
I20260809 03:35:06.453536 5882 tablet_replica.cc:442] T 4e566aadb1a3440fad2a06660bf8b237 P c747975b209c4f20ad38ed36e3f4a3a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:06.454768 5915 mvcc.cc:205] Tried to move back new op lower bound from 7316465690382598144 to 7316465690247696384. Current Snapshot: MvccSnapshot[applied={T|T < 7316465690382598144}]
I20260809 03:35:06.455094 5907 tablet_replica.cc:439] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb967b49e75e4747912973867608459d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } } }
I20260809 03:35:06.455224 5907 tablet_replica.cc:442] T 4e566aadb1a3440fad2a06660bf8b237 P eb967b49e75e4747912973867608459d: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:35:06.456585 5880 tablet_replica.cc:439] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb967b49e75e4747912973867608459d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } } }
I20260809 03:35:06.456709 5880 tablet_replica.cc:442] T 4e566aadb1a3440fad2a06660bf8b237 P a3e45c808a6846548792c864a23f7082: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:06.458290 5914 mvcc.cc:205] Tried to move back new op lower bound from 7316465690382598144 to 7316465690247696384. Current Snapshot: MvccSnapshot[applied={T|T < 7316465690382598144 or (T in {7316465690382598144})}]
I20260809 03:35:06.463470 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.459202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42444 (local address 127.1.224.65:39897)
0809 03:35:06.459340 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.459345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.459709 (+ 364us) server_negotiation.cc:407] Connection header received
0809 03:35:06.459944 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.459947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.460013 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.460127 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:06.460415 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.461002 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.461921 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.462183 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.462297 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.462597 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.463061 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.463192 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.463268 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.463325 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:35:06.476557 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.472469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:56373 (local address 127.1.224.66:34829)
0809 03:35:06.472632 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.472637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.472924 (+ 287us) server_negotiation.cc:407] Connection header received
0809 03:35:06.473110 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.473114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.473172 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.473258 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:06.473512 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.474308 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.475223 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.475582 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.475682 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.475983 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.476073 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.476147 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.476309 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.476374 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:35:06.476568 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.472369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:34829 (local address 127.1.224.65:56373)
0809 03:35:06.472828 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.472841 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.472993 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.473274 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.473276 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.473283 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:35:06.473405 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.473410 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.474428 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.474431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.475102 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.475111 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.475695 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.475697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.475772 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.475823 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.476188 (+ 365us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.476196 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.476271 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.476343 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":148,"threads_started":1}
I20260809 03:35:06.484397 5493 raft_consensus.cc:1261] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Refusing update from remote peer a3e45c808a6846548792c864a23f7082: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:06.484834 5713 raft_consensus.cc:1261] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Refusing update from remote peer a3e45c808a6846548792c864a23f7082: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:06.485114 5888 consensus_queue.cc:1035] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:06.485304 5880 consensus_queue.cc:1035] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:06.491761 5913 mvcc.cc:205] Tried to move back new op lower bound from 7316465690554486784 to 7316465689577123840. Current Snapshot: MvccSnapshot[applied={T|T < 7316465690554486784}]
I20260809 03:35:06.511451 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.507454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39897 (local address 127.1.224.66:49507)
0809 03:35:06.507808 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.507822 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.507919 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.508183 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.508185 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.508193 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:35:06.508323 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.508329 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.509379 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.509384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.510120 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.510130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.510619 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.510622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.510706 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.510759 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.511108 (+ 349us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.511117 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.511208 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.511293 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":134,"threads_started":1}
I20260809 03:35:06.511436 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.507506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:49507 (local address 127.1.224.65:39897)
0809 03:35:06.507674 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.507680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.507896 (+ 216us) server_negotiation.cc:407] Connection header received
0809 03:35:06.507993 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.507996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.508050 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.508129 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:06.508442 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.509260 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.510253 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.510493 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.510609 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.510922 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.511012 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.511104 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.511222 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.511282 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 03:35:06.537945 4922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:06.538166 4922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:06.541219 4922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6: Sending DeleteTablet for 3 replicas of tablet de07d26cbbec4139a9e8652cda09cef8
I20260809 03:35:06.542207 5695 tablet_service.cc:1514] Processing DeleteTablet for tablet de07d26cbbec4139a9e8652cda09cef8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:06 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:58394
I20260809 03:35:06.542887 5943 tablet_replica.cc:330] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0: stopping tablet replica
I20260809 03:35:06.543200 5196 tablet_service.cc:1514] Processing DeleteTablet for tablet de07d26cbbec4139a9e8652cda09cef8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:06 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:42438
I20260809 03:35:06.543272 5943 raft_consensus.cc:2229] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:06.543516 5943 raft_consensus.cc:2258] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:06.543792 5460 tablet_service.cc:1514] Processing DeleteTablet for tablet de07d26cbbec4139a9e8652cda09cef8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:06 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:51520
I20260809 03:35:06.545104 5943 ts_tablet_manager.cc:1905] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:06.545425 5944 tablet_replica.cc:330] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082: stopping tablet replica
I20260809 03:35:06.545365 4922 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 03:35:06.545847 5945 tablet_replica.cc:330] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d: stopping tablet replica
W20260809 03:35:06.545852 4922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:06.546249 5945 raft_consensus.cc:2229] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:06.546629 5944 raft_consensus.cc:2229] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:06.546886 5945 raft_consensus.cc:2258] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:06.547335 5944 raft_consensus.cc:2258] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:06.548866 5945 ts_tablet_manager.cc:1905] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:06.551539 5944 ts_tablet_manager.cc:1905] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:06.552040 5460 tablet_service.cc:1467] Processing CreateTablet for tablet 4e814421e4794745bd8839e673761c9f (DEFAULT_TABLE table=test-table [id=53ce29e7b3e7494faddf45d3f6d9691c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:06.552330 5460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e814421e4794745bd8839e673761c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:06.551699 5196 tablet_service.cc:1467] Processing CreateTablet for tablet 4e814421e4794745bd8839e673761c9f (DEFAULT_TABLE table=test-table [id=53ce29e7b3e7494faddf45d3f6d9691c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:06.552768 5196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e814421e4794745bd8839e673761c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:06.555500 5869 tablet_bootstrap.cc:492] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082: Bootstrap starting.
I20260809 03:35:06.556092 5872 tablet_bootstrap.cc:492] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d: Bootstrap starting.
I20260809 03:35:06.556568 5869 tablet_bootstrap.cc:654] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:06.557554 5872 tablet_bootstrap.cc:654] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:06.555524 5695 tablet_service.cc:1467] Processing CreateTablet for tablet 4e814421e4794745bd8839e673761c9f (DEFAULT_TABLE table=test-table [id=53ce29e7b3e7494faddf45d3f6d9691c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:06.559082 5695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e814421e4794745bd8839e673761c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:06.561776 5870 tablet_bootstrap.cc:492] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0: Bootstrap starting.
I20260809 03:35:06.562716 5870 tablet_bootstrap.cc:654] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:06.564850 5944 ts_tablet_manager.cc:1918] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:06.565286 5872 tablet_bootstrap.cc:492] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d: No bootstrap required, opened a new log
I20260809 03:35:06.565402 5872 ts_tablet_manager.cc:1397] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20260809 03:35:06.565883 5872 raft_consensus.cc:348] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.566023 5872 raft_consensus.cc:374] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:06.566064 5872 raft_consensus.cc:729] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb967b49e75e4747912973867608459d, State: Initialized, Role: FOLLOWER
I20260809 03:35:06.566216 5872 consensus_queue.cc:260] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.566329 5943 ts_tablet_manager.cc:1918] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:06.566478 5943 log.cc:1196] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/de07d26cbbec4139a9e8652cda09cef8
I20260809 03:35:06.566592 5872 ts_tablet_manager.cc:1428] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260809 03:35:06.567373 5945 ts_tablet_manager.cc:1918] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:06.568297 5869 tablet_bootstrap.cc:492] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082: No bootstrap required, opened a new log
I20260809 03:35:06.568621 5869 ts_tablet_manager.cc:1397] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.002s
I20260809 03:35:06.569115 5869 raft_consensus.cc:348] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.569229 5869 raft_consensus.cc:374] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:06.569262 5869 raft_consensus.cc:729] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3e45c808a6846548792c864a23f7082, State: Initialized, Role: FOLLOWER
I20260809 03:35:06.569403 5869 consensus_queue.cc:260] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.569691 5869 ts_tablet_manager.cc:1428] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:06.570124 5943 ts_tablet_manager.cc:1939] T de07d26cbbec4139a9e8652cda09cef8 P c747975b209c4f20ad38ed36e3f4a3a0: Deleting consensus metadata
I20260809 03:35:06.571506 4884 catalog_manager.cc:4841] TS c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729): tablet de07d26cbbec4139a9e8652cda09cef8 (table test-table [id=8953746d04d14f7d9f0f152d3bcc4186]) successfully deleted
I20260809 03:35:06.572286 5870 tablet_bootstrap.cc:492] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0: No bootstrap required, opened a new log
I20260809 03:35:06.572373 5870 ts_tablet_manager.cc:1397] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260809 03:35:06.572818 5870 raft_consensus.cc:348] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.572949 5870 raft_consensus.cc:374] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:06.572983 5870 raft_consensus.cc:729] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c747975b209c4f20ad38ed36e3f4a3a0, State: Initialized, Role: FOLLOWER
I20260809 03:35:06.573097 5870 consensus_queue.cc:260] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.573352 5870 ts_tablet_manager.cc:1428] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:06.573576 5944 log.cc:1196] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/wal/wals/de07d26cbbec4139a9e8652cda09cef8
I20260809 03:35:06.568437 5945 log.cc:1196] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/de07d26cbbec4139a9e8652cda09cef8
I20260809 03:35:06.574059 5945 ts_tablet_manager.cc:1939] T de07d26cbbec4139a9e8652cda09cef8 P eb967b49e75e4747912973867608459d: Deleting consensus metadata
I20260809 03:35:06.574481 5944 ts_tablet_manager.cc:1939] T de07d26cbbec4139a9e8652cda09cef8 P a3e45c808a6846548792c864a23f7082: Deleting consensus metadata
I20260809 03:35:06.575790 4891 catalog_manager.cc:4841] TS eb967b49e75e4747912973867608459d (127.1.224.66:34829): tablet de07d26cbbec4139a9e8652cda09cef8 (table test-table [id=8953746d04d14f7d9f0f152d3bcc4186]) successfully deleted
I20260809 03:35:06.577267 4891 catalog_manager.cc:4841] TS a3e45c808a6846548792c864a23f7082 (127.1.224.65:39897): tablet de07d26cbbec4139a9e8652cda09cef8 (table test-table [id=8953746d04d14f7d9f0f152d3bcc4186]) successfully deleted
I20260809 03:35:06.583340 5924 raft_consensus.cc:482] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:06.583480 5924 raft_consensus.cc:504] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.583925 5924 leader_election.cc:288] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3e45c808a6846548792c864a23f7082 (127.1.224.65:39897), c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729)
I20260809 03:35:06.584415 5232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e814421e4794745bd8839e673761c9f" candidate_uuid: "eb967b49e75e4747912973867608459d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3e45c808a6846548792c864a23f7082" is_pre_election: true
I20260809 03:35:06.584602 5232 raft_consensus.cc:2454] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb967b49e75e4747912973867608459d in term 0.
I20260809 03:35:06.584762 5713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e814421e4794745bd8839e673761c9f" candidate_uuid: "eb967b49e75e4747912973867608459d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" is_pre_election: true
I20260809 03:35:06.584915 5713 raft_consensus.cc:2454] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb967b49e75e4747912973867608459d in term 0.
I20260809 03:35:06.585073 5364 leader_election.cc:302] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3e45c808a6846548792c864a23f7082, eb967b49e75e4747912973867608459d; no voters:
I20260809 03:35:06.585345 5924 raft_consensus.cc:2781] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:06.585414 5924 raft_consensus.cc:482] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:06.585453 5924 raft_consensus.cc:3037] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.586751 5924 raft_consensus.cc:504] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.587155 5924 leader_election.cc:288] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [CANDIDATE]: Term 1 election: Requested vote from peers a3e45c808a6846548792c864a23f7082 (127.1.224.65:39897), c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729)
I20260809 03:35:06.587549 5232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e814421e4794745bd8839e673761c9f" candidate_uuid: "eb967b49e75e4747912973867608459d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3e45c808a6846548792c864a23f7082"
I20260809 03:35:06.587586 5713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e814421e4794745bd8839e673761c9f" candidate_uuid: "eb967b49e75e4747912973867608459d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c747975b209c4f20ad38ed36e3f4a3a0"
I20260809 03:35:06.587687 5232 raft_consensus.cc:3037] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.587713 5713 raft_consensus.cc:3037] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:06.589286 5232 raft_consensus.cc:2454] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb967b49e75e4747912973867608459d in term 1.
I20260809 03:35:06.589701 5713 raft_consensus.cc:2454] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb967b49e75e4747912973867608459d in term 1.
I20260809 03:35:06.589797 5364 leader_election.cc:302] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3e45c808a6846548792c864a23f7082, eb967b49e75e4747912973867608459d; no voters:
I20260809 03:35:06.590065 5924 raft_consensus.cc:2781] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:06.590152 5924 raft_consensus.cc:686] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 1 LEADER]: Becoming Leader. State: Replica: eb967b49e75e4747912973867608459d, State: Running, Role: LEADER
I20260809 03:35:06.590288 5924 consensus_queue.cc:237] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:06.592343 4922 catalog_manager.cc:5495] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d reported cstate change: term changed from 0 to 1, leader changed from <none> to eb967b49e75e4747912973867608459d (127.1.224.66). New cstate: current_term: 1 leader_uuid: "eb967b49e75e4747912973867608459d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } health_report { overall_health: UNKNOWN } } }
I20260809 03:35:06.612879 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.609318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51536 (local address 127.1.224.66:34829)
0809 03:35:06.609427 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.609436 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.609448 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:35:06.609576 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.609578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.609623 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.609685 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:06.609958 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.610459 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.611667 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.611896 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.611979 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.612289 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.612385 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.612455 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.612641 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.612697 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 03:35:06.623402 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.620029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:47793 (local address 127.1.224.66:34829)
0809 03:35:06.620160 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:06.620164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:06.620246 (+ 82us) server_negotiation.cc:407] Connection header received
0809 03:35:06.620414 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:06.620417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:06.620510 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:06.620590 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:06.620894 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.621407 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.622337 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.622561 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.622667 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.622953 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.623029 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.623102 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:06.623193 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 03:35:06.623242 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":47}
I20260809 03:35:06.623405 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:06.619962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:34829 (local address 127.1.224.66:47793)
0809 03:35:06.620166 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:06.620181 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:06.620299 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:06.620597 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:06.620599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:06.620606 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:35:06.620753 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.620760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.621526 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.621532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.622223 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:06.622232 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:06.622665 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:06.622668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:06.622805 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:06.622837 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:35:06.623121 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:35:06.623126 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:35:06.623186 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 03:35:06.623276 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":56}
I20260809 03:35:06.629767 5713 raft_consensus.cc:1261] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Refusing update from remote peer eb967b49e75e4747912973867608459d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:06.630215 5924 consensus_queue.cc:1035] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:06.635322 5232 raft_consensus.cc:1261] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Refusing update from remote peer eb967b49e75e4747912973867608459d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:06.636380 5881 consensus_queue.cc:1035] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:35:06.649737 5913 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 03:35:08.212296 4922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:08.212463 4922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:08.215189 4922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6: Sending DeleteTablet for 3 replicas of tablet 4e814421e4794745bd8839e673761c9f
I20260809 03:35:08.215911 5196 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e814421e4794745bd8839e673761c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:08 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:42438
I20260809 03:35:08.215914 5460 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e814421e4794745bd8839e673761c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:08 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:51520
I20260809 03:35:08.216065 5695 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e814421e4794745bd8839e673761c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:08 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:58394
I20260809 03:35:08.216338 5996 tablet_replica.cc:330] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082: stopping tablet replica
I20260809 03:35:08.216470 5994 tablet_replica.cc:330] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d: stopping tablet replica
I20260809 03:35:08.216522 5996 raft_consensus.cc:2229] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:08.216630 5994 raft_consensus.cc:2229] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:08.216670 5996 raft_consensus.cc:2258] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:08.217058 5994 raft_consensus.cc:2258] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:08.217372 6000 tablet_replica.cc:330] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0: stopping tablet replica
I20260809 03:35:08.217454 5996 ts_tablet_manager.cc:1905] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:08.217514 6000 raft_consensus.cc:2229] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:08.217643 6000 raft_consensus.cc:2258] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:08.217993 5994 ts_tablet_manager.cc:1905] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:08.218339 6000 ts_tablet_manager.cc:1905] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:08.221077 5994 ts_tablet_manager.cc:1918] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:35:08.221225 5994 log.cc:1196] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/4e814421e4794745bd8839e673761c9f
I20260809 03:35:08.221243 6000 ts_tablet_manager.cc:1918] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:35:08.221275 5996 ts_tablet_manager.cc:1918] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:35:08.221315 6000 log.cc:1196] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/4e814421e4794745bd8839e673761c9f
I20260809 03:35:08.221365 5996 log.cc:1196] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/wal/wals/4e814421e4794745bd8839e673761c9f
I20260809 03:35:08.221701 5994 ts_tablet_manager.cc:1939] T 4e814421e4794745bd8839e673761c9f P eb967b49e75e4747912973867608459d: Deleting consensus metadata
I20260809 03:35:08.221740 6000 ts_tablet_manager.cc:1939] T 4e814421e4794745bd8839e673761c9f P c747975b209c4f20ad38ed36e3f4a3a0: Deleting consensus metadata
I20260809 03:35:08.221768 5996 ts_tablet_manager.cc:1939] T 4e814421e4794745bd8839e673761c9f P a3e45c808a6846548792c864a23f7082: Deleting consensus metadata
I20260809 03:35:08.222620 4884 catalog_manager.cc:4841] TS c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729): tablet 4e814421e4794745bd8839e673761c9f (table test-table [id=53ce29e7b3e7494faddf45d3f6d9691c]) successfully deleted
I20260809 03:35:08.222870 4891 catalog_manager.cc:4841] TS a3e45c808a6846548792c864a23f7082 (127.1.224.65:39897): tablet 4e814421e4794745bd8839e673761c9f (table test-table [id=53ce29e7b3e7494faddf45d3f6d9691c]) successfully deleted
I20260809 03:35:08.223003 4891 catalog_manager.cc:4841] TS eb967b49e75e4747912973867608459d (127.1.224.66:34829): tablet 4e814421e4794745bd8839e673761c9f (table test-table [id=53ce29e7b3e7494faddf45d3f6d9691c]) successfully deleted
Aug 09 03:35:08 dist-test-slave-jq4t krb5kdc[4832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246508, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:35:08 dist-test-slave-jq4t krb5kdc[4832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246508, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:08.253365 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.238881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40502 (local address 127.1.224.126:36931)
0809 03:35:08.239209 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:08.239215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:08.239232 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:35:08.239284 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:08.239287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:08.239331 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:08.239412 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:08.240135 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.240786 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.241617 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.241849 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.241927 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.249601 (+ 7674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:08.249606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:08.249608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:08.249651 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:08.251399 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:08.251879 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:08.251898 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:08.251900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:08.252002 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:08.252290 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:08.252292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:08.252293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:08.252641 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:08.252760 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:08.252999 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 03:35:08.253124 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":108,"threads_started":1}
I20260809 03:35:08.256752 4922 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:08.257278 4922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:08.262552 5196 tablet_service.cc:1467] Processing CreateTablet for tablet a54a3cabe23f4221a0a0265ef2f4d89f (DEFAULT_TABLE table=test-table [id=6d7b78af1b784323a1d10ff5d0a1882a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:08.262620 5460 tablet_service.cc:1467] Processing CreateTablet for tablet a54a3cabe23f4221a0a0265ef2f4d89f (DEFAULT_TABLE table=test-table [id=6d7b78af1b784323a1d10ff5d0a1882a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:08.262653 5695 tablet_service.cc:1467] Processing CreateTablet for tablet a54a3cabe23f4221a0a0265ef2f4d89f (DEFAULT_TABLE table=test-table [id=6d7b78af1b784323a1d10ff5d0a1882a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:08.262923 5695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54a3cabe23f4221a0a0265ef2f4d89f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:08.262923 5196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54a3cabe23f4221a0a0265ef2f4d89f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:08.262946 5460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54a3cabe23f4221a0a0265ef2f4d89f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:08.265261 6014 tablet_bootstrap.cc:492] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d: Bootstrap starting.
I20260809 03:35:08.265751 6015 tablet_bootstrap.cc:492] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0: Bootstrap starting.
I20260809 03:35:08.266319 6014 tablet_bootstrap.cc:654] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:08.266695 6015 tablet_bootstrap.cc:654] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:08.267414 6017 tablet_bootstrap.cc:492] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082: Bootstrap starting.
I20260809 03:35:08.267603 6014 tablet_bootstrap.cc:492] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d: No bootstrap required, opened a new log
I20260809 03:35:08.267717 6014 ts_tablet_manager.cc:1397] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:08.268206 6014 raft_consensus.cc:348] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.268344 6014 raft_consensus.cc:374] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:08.268388 6014 raft_consensus.cc:729] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb967b49e75e4747912973867608459d, State: Initialized, Role: FOLLOWER
I20260809 03:35:08.268586 6014 consensus_queue.cc:260] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.268958 6015 tablet_bootstrap.cc:492] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0: No bootstrap required, opened a new log
I20260809 03:35:08.269001 6014 ts_tablet_manager.cc:1428] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 03:35:08.269074 6015 ts_tablet_manager.cc:1397] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:08.269913 6015 raft_consensus.cc:348] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.270021 6015 raft_consensus.cc:374] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:08.270045 6015 raft_consensus.cc:729] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c747975b209c4f20ad38ed36e3f4a3a0, State: Initialized, Role: FOLLOWER
I20260809 03:35:08.270190 6015 consensus_queue.cc:260] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.270625 6015 ts_tablet_manager.cc:1428] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:08.270915 6017 tablet_bootstrap.cc:654] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:08.272372 6017 tablet_bootstrap.cc:492] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082: No bootstrap required, opened a new log
I20260809 03:35:08.272529 6017 ts_tablet_manager.cc:1397] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260809 03:35:08.272989 6017 raft_consensus.cc:348] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.273110 6017 raft_consensus.cc:374] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:08.273156 6017 raft_consensus.cc:729] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3e45c808a6846548792c864a23f7082, State: Initialized, Role: FOLLOWER
I20260809 03:35:08.273298 6017 consensus_queue.cc:260] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.273742 6017 ts_tablet_manager.cc:1428] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:08.553179 6021 raft_consensus.cc:482] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:08.553354 6021 raft_consensus.cc:504] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.554649 6021 leader_election.cc:288] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3e45c808a6846548792c864a23f7082 (127.1.224.65:39897), eb967b49e75e4747912973867608459d (127.1.224.66:34829)
I20260809 03:35:08.559227 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.555117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:41355 (local address 127.1.224.65:39897)
0809 03:35:08.555509 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:08.555531 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:08.555552 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:35:08.555717 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:08.555722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:08.555782 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:08.556263 (+ 481us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:08.556378 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.556989 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.558077 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.558574 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.558712 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.558772 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:08.558965 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 03:35:08.559045 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":172,"threads_started":1}
I20260809 03:35:08.559239 6025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.555048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39897 (local address 127.1.224.67:41355)
0809 03:35:08.555414 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:08.555430 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:08.555586 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:08.555915 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:08.555920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:08.555946 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:08.556135 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:08.556141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.557108 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.557115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:08.557940 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:08.557948 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.558712 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.558716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:08.558819 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.558845 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 03:35:08.558931 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 03:35:08.559014 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"spinlock_wait_cycles":1920,"thread_start_us":135,"threads_started":1}
I20260809 03:35:08.559252 6032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.555704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:53135 (local address 127.1.224.66:34829)
0809 03:35:08.556023 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:08.556030 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:08.556043 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:08.556092 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:08.556094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:08.556136 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:08.556212 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:08.556543 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.557074 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.558123 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.558614 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.558753 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.558813 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:08.559024 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 03:35:08.559104 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":151,"threads_started":1}
I20260809 03:35:08.559270 6031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.555521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:34829 (local address 127.1.224.67:53135)
0809 03:35:08.555895 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:08.555908 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:08.555990 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:08.556249 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:08.556252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:08.556271 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:08.556381 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:08.556387 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.557199 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.557204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:08.557982 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:08.557992 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.558772 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.558776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:08.558882 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.558901 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 03:35:08.558993 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 03:35:08.559065 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":105,"threads_started":1}
I20260809 03:35:08.559854 5493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54a3cabe23f4221a0a0265ef2f4d89f" candidate_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb967b49e75e4747912973867608459d" is_pre_election: true
I20260809 03:35:08.560060 5493 raft_consensus.cc:2454] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c747975b209c4f20ad38ed36e3f4a3a0 in term 0.
I20260809 03:35:08.560189 5228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54a3cabe23f4221a0a0265ef2f4d89f" candidate_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3e45c808a6846548792c864a23f7082" is_pre_election: true
I20260809 03:35:08.560345 5228 raft_consensus.cc:2454] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c747975b209c4f20ad38ed36e3f4a3a0 in term 0.
I20260809 03:35:08.560453 5627 leader_election.cc:302] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c747975b209c4f20ad38ed36e3f4a3a0, eb967b49e75e4747912973867608459d; no voters:
I20260809 03:35:08.560788 6021 raft_consensus.cc:2781] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:08.560847 6021 raft_consensus.cc:482] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:08.560875 6021 raft_consensus.cc:3037] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:08.561867 6021 raft_consensus.cc:504] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.562165 6021 leader_election.cc:288] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [CANDIDATE]: Term 1 election: Requested vote from peers a3e45c808a6846548792c864a23f7082 (127.1.224.65:39897), eb967b49e75e4747912973867608459d (127.1.224.66:34829)
I20260809 03:35:08.562543 5228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54a3cabe23f4221a0a0265ef2f4d89f" candidate_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3e45c808a6846548792c864a23f7082"
I20260809 03:35:08.562609 5493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54a3cabe23f4221a0a0265ef2f4d89f" candidate_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb967b49e75e4747912973867608459d"
I20260809 03:35:08.562668 5228 raft_consensus.cc:3037] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:08.562767 5493 raft_consensus.cc:3037] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:08.563647 5228 raft_consensus.cc:2454] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c747975b209c4f20ad38ed36e3f4a3a0 in term 1.
I20260809 03:35:08.563800 5493 raft_consensus.cc:2454] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c747975b209c4f20ad38ed36e3f4a3a0 in term 1.
I20260809 03:35:08.563943 5627 leader_election.cc:302] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3e45c808a6846548792c864a23f7082, c747975b209c4f20ad38ed36e3f4a3a0; no voters:
I20260809 03:35:08.564167 6021 raft_consensus.cc:2781] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:08.564378 6021 raft_consensus.cc:686] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 LEADER]: Becoming Leader. State: Replica: c747975b209c4f20ad38ed36e3f4a3a0, State: Running, Role: LEADER
I20260809 03:35:08.564664 6021 consensus_queue.cc:237] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.567368 4922 catalog_manager.cc:5495] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67). New cstate: current_term: 1 leader_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } health_report { overall_health: HEALTHY } } }
I20260809 03:35:08.646986 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.642589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58406 (local address 127.1.224.67:37729)
0809 03:35:08.642883 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:08.642895 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:08.642908 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:08.642988 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:08.642991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:08.643049 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:08.643166 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:08.643453 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.644017 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.645341 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.645600 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.645688 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.645996 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:08.646507 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:08.646600 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:08.646694 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 03:35:08.646753 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":130,"threads_started":1}
I20260809 03:35:08.658569 6052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.654588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:34829 (local address 127.1.224.67:34393)
0809 03:35:08.654981 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:08.654999 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:08.655143 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:08.655448 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:08.655451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:08.655459 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:35:08.655607 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:08.655614 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.656451 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.656455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:08.657176 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:08.657186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.657679 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.657683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:08.657797 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.657856 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:35:08.658191 (+ 335us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:35:08.658201 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 03:35:08.658295 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 03:35:08.658385 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":136,"threads_started":1}
I20260809 03:35:08.658563 6032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.654853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:34393 (local address 127.1.224.66:34829)
0809 03:35:08.654985 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:08.654992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:08.655099 (+ 107us) server_negotiation.cc:407] Connection header received
0809 03:35:08.655272 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:08.655276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:08.655342 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:08.655433 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:08.655778 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.656319 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.657318 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.657561 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.657676 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.658014 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:08.658100 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:08.658183 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:08.658307 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 03:35:08.658375 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:35:08.665211 5493 raft_consensus.cc:1261] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Refusing update from remote peer c747975b209c4f20ad38ed36e3f4a3a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:08.665247 5228 raft_consensus.cc:1261] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Refusing update from remote peer c747975b209c4f20ad38ed36e3f4a3a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:08.665735 6021 consensus_queue.cc:1035] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:08.665959 6033 consensus_queue.cc:1035] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:08.683779 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.680167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:59759 (local address 127.1.224.67:37729)
0809 03:35:08.680312 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:08.680317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:08.680559 (+ 242us) server_negotiation.cc:407] Connection header received
0809 03:35:08.680681 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:08.680684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:08.680733 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:08.680816 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:08.681064 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.681583 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.682518 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.682853 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.682972 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.683247 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:08.683362 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:08.683442 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:08.683594 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 03:35:08.683658 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:35:08.683825 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.680072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:37729 (local address 127.1.224.66:59759)
0809 03:35:08.680430 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:08.680445 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:08.680595 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:08.680829 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:08.680832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:08.680841 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:35:08.680951 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:08.680957 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.681700 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.681704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:08.682381 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:08.682392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.682983 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.682986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:08.683089 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.683123 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:35:08.683478 (+ 355us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:35:08.683488 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 03:35:08.683565 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 03:35:08.683685 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":113,"threads_started":1}
I20260809 03:35:08.707114 4924 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:08.707307 4924 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:08.709649 4924 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6: Sending DeleteTablet for 3 replicas of tablet a54a3cabe23f4221a0a0265ef2f4d89f
I20260809 03:35:08.710309 5196 tablet_service.cc:1514] Processing DeleteTablet for tablet a54a3cabe23f4221a0a0265ef2f4d89f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:08 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:42438
I20260809 03:35:08.710502 5695 tablet_service.cc:1514] Processing DeleteTablet for tablet a54a3cabe23f4221a0a0265ef2f4d89f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:08 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:58394
I20260809 03:35:08.710713 6000 tablet_replica.cc:330] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0: stopping tablet replica
I20260809 03:35:08.710745 5996 tablet_replica.cc:330] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082: stopping tablet replica
I20260809 03:35:08.710862 6000 raft_consensus.cc:2229] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:08.710880 5996 raft_consensus.cc:2229] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:08.711000 5996 raft_consensus.cc:2258] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:08.711198 6000 raft_consensus.cc:2258] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:08.711334 5460 tablet_service.cc:1514] Processing DeleteTablet for tablet a54a3cabe23f4221a0a0265ef2f4d89f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:08 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:51520
I20260809 03:35:08.711535 5994 tablet_replica.cc:330] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d: stopping tablet replica
I20260809 03:35:08.711663 5994 raft_consensus.cc:2229] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:08.711796 5994 raft_consensus.cc:2258] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:08.712064 6000 ts_tablet_manager.cc:1905] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:08.712647 5996 ts_tablet_manager.cc:1905] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:08.714872 6000 ts_tablet_manager.cc:1918] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:08.714963 6000 log.cc:1196] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/a54a3cabe23f4221a0a0265ef2f4d89f
I20260809 03:35:08.715260 6000 ts_tablet_manager.cc:1939] T a54a3cabe23f4221a0a0265ef2f4d89f P c747975b209c4f20ad38ed36e3f4a3a0: Deleting consensus metadata
I20260809 03:35:08.715385 5996 ts_tablet_manager.cc:1918] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:08.715467 5996 log.cc:1196] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-0/wal/wals/a54a3cabe23f4221a0a0265ef2f4d89f
I20260809 03:35:08.715728 5996 ts_tablet_manager.cc:1939] T a54a3cabe23f4221a0a0265ef2f4d89f P a3e45c808a6846548792c864a23f7082: Deleting consensus metadata
I20260809 03:35:08.716084 4884 catalog_manager.cc:4841] TS c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729): tablet a54a3cabe23f4221a0a0265ef2f4d89f (table test-table [id=6d7b78af1b784323a1d10ff5d0a1882a]) successfully deleted
I20260809 03:35:08.716212 4924 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 03:35:08.716503 4891 catalog_manager.cc:4841] TS a3e45c808a6846548792c864a23f7082 (127.1.224.65:39897): tablet a54a3cabe23f4221a0a0265ef2f4d89f (table test-table [id=6d7b78af1b784323a1d10ff5d0a1882a]) successfully deleted
W20260809 03:35:08.716730 4924 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:08.716931 5994 ts_tablet_manager.cc:1905] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:08.719630 5994 ts_tablet_manager.cc:1918] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:08.719733 5994 log.cc:1196] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/a54a3cabe23f4221a0a0265ef2f4d89f
I20260809 03:35:08.719993 5994 ts_tablet_manager.cc:1939] T a54a3cabe23f4221a0a0265ef2f4d89f P eb967b49e75e4747912973867608459d: Deleting consensus metadata
I20260809 03:35:08.720908 4891 catalog_manager.cc:4841] TS eb967b49e75e4747912973867608459d (127.1.224.66:34829): tablet a54a3cabe23f4221a0a0265ef2f4d89f (table test-table [id=6d7b78af1b784323a1d10ff5d0a1882a]) successfully deleted
I20260809 03:35:08.721927 5460 tablet_service.cc:1467] Processing CreateTablet for tablet 2c95ee47da17472296cd48e72b2e2c27 (DEFAULT_TABLE table=test-table [id=fbeff45ae4dc404890838cae9feae876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:08.721936 5196 tablet_service.cc:1467] Processing CreateTablet for tablet 2c95ee47da17472296cd48e72b2e2c27 (DEFAULT_TABLE table=test-table [id=fbeff45ae4dc404890838cae9feae876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:08.722086 5695 tablet_service.cc:1467] Processing CreateTablet for tablet 2c95ee47da17472296cd48e72b2e2c27 (DEFAULT_TABLE table=test-table [id=fbeff45ae4dc404890838cae9feae876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:08.722190 5196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c95ee47da17472296cd48e72b2e2c27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:08.722193 5460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c95ee47da17472296cd48e72b2e2c27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:08.722321 5695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c95ee47da17472296cd48e72b2e2c27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:08.724587 6015 tablet_bootstrap.cc:492] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0: Bootstrap starting.
I20260809 03:35:08.724677 6014 tablet_bootstrap.cc:492] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d: Bootstrap starting.
I20260809 03:35:08.725399 6014 tablet_bootstrap.cc:654] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:08.725467 6015 tablet_bootstrap.cc:654] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:08.725905 6017 tablet_bootstrap.cc:492] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082: Bootstrap starting.
I20260809 03:35:08.726676 6015 tablet_bootstrap.cc:492] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0: No bootstrap required, opened a new log
I20260809 03:35:08.726764 6015 ts_tablet_manager.cc:1397] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:35:08.726841 6017 tablet_bootstrap.cc:654] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:08.727270 6015 raft_consensus.cc:348] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.727382 6015 raft_consensus.cc:374] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:08.727415 6015 raft_consensus.cc:729] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c747975b209c4f20ad38ed36e3f4a3a0, State: Initialized, Role: FOLLOWER
I20260809 03:35:08.727571 6015 consensus_queue.cc:260] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.727893 6015 ts_tablet_manager.cc:1428] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:08.728200 6014 tablet_bootstrap.cc:492] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d: No bootstrap required, opened a new log
I20260809 03:35:08.728279 6014 ts_tablet_manager.cc:1397] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 03:35:08.728757 6014 raft_consensus.cc:348] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.728865 6014 raft_consensus.cc:374] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:08.728888 6014 raft_consensus.cc:729] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb967b49e75e4747912973867608459d, State: Initialized, Role: FOLLOWER
I20260809 03:35:08.729031 6014 consensus_queue.cc:260] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.729307 6014 ts_tablet_manager.cc:1428] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:08.731539 6017 tablet_bootstrap.cc:492] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082: No bootstrap required, opened a new log
I20260809 03:35:08.731655 6017 ts_tablet_manager.cc:1397] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260809 03:35:08.732208 6017 raft_consensus.cc:348] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.732350 6017 raft_consensus.cc:374] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:08.732388 6017 raft_consensus.cc:729] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3e45c808a6846548792c864a23f7082, State: Initialized, Role: FOLLOWER
I20260809 03:35:08.732877 6017 consensus_queue.cc:260] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.733218 6017 ts_tablet_manager.cc:1428] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260809 03:35:08.734093 6024 raft_consensus.cc:482] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:08.734251 6024 raft_consensus.cc:504] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.734791 6024 leader_election.cc:288] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb967b49e75e4747912973867608459d (127.1.224.66:34829), c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729)
I20260809 03:35:08.735252 5745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c95ee47da17472296cd48e72b2e2c27" candidate_uuid: "a3e45c808a6846548792c864a23f7082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" is_pre_election: true
I20260809 03:35:08.735397 5745 raft_consensus.cc:2454] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3e45c808a6846548792c864a23f7082 in term 0.
I20260809 03:35:08.735394 5493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c95ee47da17472296cd48e72b2e2c27" candidate_uuid: "a3e45c808a6846548792c864a23f7082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb967b49e75e4747912973867608459d" is_pre_election: true
I20260809 03:35:08.735555 5493 raft_consensus.cc:2454] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3e45c808a6846548792c864a23f7082 in term 0.
I20260809 03:35:08.735693 5107 leader_election.cc:302] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3e45c808a6846548792c864a23f7082, c747975b209c4f20ad38ed36e3f4a3a0; no voters:
I20260809 03:35:08.735957 6024 raft_consensus.cc:2781] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:08.736027 6024 raft_consensus.cc:482] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:08.736073 6024 raft_consensus.cc:3037] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:08.737519 6024 raft_consensus.cc:504] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.737916 6024 leader_election.cc:288] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [CANDIDATE]: Term 1 election: Requested vote from peers eb967b49e75e4747912973867608459d (127.1.224.66:34829), c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729)
I20260809 03:35:08.738268 5745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c95ee47da17472296cd48e72b2e2c27" candidate_uuid: "a3e45c808a6846548792c864a23f7082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c747975b209c4f20ad38ed36e3f4a3a0"
I20260809 03:35:08.738268 5493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c95ee47da17472296cd48e72b2e2c27" candidate_uuid: "a3e45c808a6846548792c864a23f7082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb967b49e75e4747912973867608459d"
I20260809 03:35:08.738394 5493 raft_consensus.cc:3037] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:08.738406 5745 raft_consensus.cc:3037] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:08.739615 5493 raft_consensus.cc:2454] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3e45c808a6846548792c864a23f7082 in term 1.
I20260809 03:35:08.739615 5745 raft_consensus.cc:2454] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3e45c808a6846548792c864a23f7082 in term 1.
I20260809 03:35:08.739916 5107 leader_election.cc:302] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3e45c808a6846548792c864a23f7082, c747975b209c4f20ad38ed36e3f4a3a0; no voters:
I20260809 03:35:08.740178 6024 raft_consensus.cc:2781] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:08.740386 6024 raft_consensus.cc:686] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [term 1 LEADER]: Becoming Leader. State: Replica: a3e45c808a6846548792c864a23f7082, State: Running, Role: LEADER
I20260809 03:35:08.740598 6024 consensus_queue.cc:237] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } }
I20260809 03:35:08.742307 4922 catalog_manager.cc:5495] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3e45c808a6846548792c864a23f7082 (127.1.224.65). New cstate: current_term: 1 leader_uuid: "a3e45c808a6846548792c864a23f7082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e45c808a6846548792c864a23f7082" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 } health_report { overall_health: UNKNOWN } } }
I20260809 03:35:08.756568 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:08.753157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42448 (local address 127.1.224.65:39897)
0809 03:35:08.753298 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:08.753302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:08.753315 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:08.753436 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:08.753439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:08.753485 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:08.753548 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:08.753819 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.754351 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.755514 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:08.755725 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:08.755800 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:08.756075 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:08.756148 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:08.756203 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:08.756324 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 03:35:08.756374 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:35:08.763695 5493 raft_consensus.cc:1261] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Refusing update from remote peer a3e45c808a6846548792c864a23f7082: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:08.764086 5745 raft_consensus.cc:1261] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Refusing update from remote peer a3e45c808a6846548792c864a23f7082: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:08.764089 6024 consensus_queue.cc:1035] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb967b49e75e4747912973867608459d" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:08.764453 6074 consensus_queue.cc:1035] T 2c95ee47da17472296cd48e72b2e2c27 P a3e45c808a6846548792c864a23f7082 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c747975b209c4f20ad38ed36e3f4a3a0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:35:08.780136 5969 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 03:35:10.338749 4922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:10.338979 4922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:10.341780 4922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4ee89b59b9d4024b84718d215f77fc6: Sending DeleteTablet for 3 replicas of tablet 2c95ee47da17472296cd48e72b2e2c27
I20260809 03:35:10.342487 5695 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c95ee47da17472296cd48e72b2e2c27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:10 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:58394
I20260809 03:35:10.342511 5196 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c95ee47da17472296cd48e72b2e2c27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:10 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:42438
I20260809 03:35:10.342867 6143 tablet_replica.cc:330] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0: stopping tablet replica
I20260809 03:35:10.342856 5460 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c95ee47da17472296cd48e72b2e2c27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:10 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:51520
I20260809 03:35:10.342904 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 5080
I20260809 03:35:10.352527 6145 tablet_replica.cc:330] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d: stopping tablet replica
I20260809 03:35:10.352746 6145 raft_consensus.cc:2229] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:10.352909 6145 raft_consensus.cc:2258] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:10.352957 6143 raft_consensus.cc:2229] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:10.353145 6143 raft_consensus.cc:2258] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:10.353781 6145 ts_tablet_manager.cc:1905] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:10.353946 6143 ts_tablet_manager.cc:1905] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:10.356189 6145 ts_tablet_manager.cc:1918] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:35:10.356325 6145 log.cc:1196] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/2c95ee47da17472296cd48e72b2e2c27
I20260809 03:35:10.356665 6145 ts_tablet_manager.cc:1939] T 2c95ee47da17472296cd48e72b2e2c27 P eb967b49e75e4747912973867608459d: Deleting consensus metadata
I20260809 03:35:10.357522 6143 ts_tablet_manager.cc:1918] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:35:10.357550 4891 catalog_manager.cc:4841] TS eb967b49e75e4747912973867608459d (127.1.224.66:34829): tablet 2c95ee47da17472296cd48e72b2e2c27 (table test-table [id=fbeff45ae4dc404890838cae9feae876]) successfully deleted
I20260809 03:35:10.357631 6143 log.cc:1196] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/2c95ee47da17472296cd48e72b2e2c27
I20260809 03:35:10.357930 6143 ts_tablet_manager.cc:1939] T 2c95ee47da17472296cd48e72b2e2c27 P c747975b209c4f20ad38ed36e3f4a3a0: Deleting consensus metadata
I20260809 03:35:10.358770 4884 catalog_manager.cc:4841] TS c747975b209c4f20ad38ed36e3f4a3a0 (127.1.224.67:37729): tablet 2c95ee47da17472296cd48e72b2e2c27 (table test-table [id=fbeff45ae4dc404890838cae9feae876]) successfully deleted
W20260809 03:35:10.361634 4891 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:39897 (error 108)
I20260809 03:35:10.361948 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 5336
I20260809 03:35:10.371707 6151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:10.370289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39897 (local address 127.0.0.1:42452)
0809 03:35:10.371468 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:10.371562 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1085,"thread_start_us":982,"threads_started":1}
W20260809 03:35:10.372044 4891 catalog_manager.cc:4584] TS a3e45c808a6846548792c864a23f7082 (127.1.224.65:39897): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c95ee47da17472296cd48e72b2e2c27: Network error: Client connection negotiation failed: client connection to 127.1.224.65:39897: connect: Connection refused (error 111)
I20260809 03:35:10.379258 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 5596
I20260809 03:35:10.395591 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 4857
2026-08-09T03:35:10Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6938 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:10 dist-test-slave-jq4t krb5kdc[6155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:10 dist-test-slave-jq4t krb5kdc[6155](info): set up 2 sockets
Aug 09 03:35:10 dist-test-slave-jq4t krb5kdc[6155](info): commencing operation
krb5kdc: starting...
W20260809 03:35:11.658938 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:35:11 dist-test-slave-jq4t krb5kdc[6155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:35:11.792323 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:36863
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:36863
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:11.911284 6174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:11.911571 6174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:11.911613 6174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:11.915576 6174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:11.915627 6174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:11.915658 6174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:11.915674 6174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:11.915689 6174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:11.920989 6174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:36863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:36863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:11.922062 6174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:11.923079 6174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:11.932227 6190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:11.932288 6182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:11.932286 6184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:11.932346 6174 server_base.cc:1034] running on GCE node
I20260809 03:35:11.933176 6174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:11.934363 6174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:11.935552 6174 hybrid_clock.cc:648] HybridClock initialized: now 1786246511935531 us; error 33 us; skew 500 ppm
Aug 09 03:35:11 dist-test-slave-jq4t krb5kdc[6155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246511, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:11.938537 6174 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:35:11.939568 6174 webserver.cc:460] Webserver started at http://127.1.224.126:39035/ using document root <none> and password file <none>
I20260809 03:35:11.940250 6174 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:11.940353 6174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:11.940676 6174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:11.942916 6174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "ff82f9769a4944b0ae6ce44963e2b053"
format_stamp: "Formatted at 2026-08-09 03:35:11 on dist-test-slave-jq4t"
server_key: "ff7cddb8dcab108a1298e1e95e28373a"
server_key_iv: "932d924e359cc00c57309a972d4fafee"
server_key_version: "encryptionkey@0"
I20260809 03:35:11.943539 6174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "ff82f9769a4944b0ae6ce44963e2b053"
format_stamp: "Formatted at 2026-08-09 03:35:11 on dist-test-slave-jq4t"
server_key: "ff7cddb8dcab108a1298e1e95e28373a"
server_key_iv: "932d924e359cc00c57309a972d4fafee"
server_key_version: "encryptionkey@0"
I20260809 03:35:11.947544 6174 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 03:35:11.950125 6204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:11.951398 6174 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:35:11.951542 6174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "ff82f9769a4944b0ae6ce44963e2b053"
format_stamp: "Formatted at 2026-08-09 03:35:11 on dist-test-slave-jq4t"
server_key: "ff7cddb8dcab108a1298e1e95e28373a"
server_key_iv: "932d924e359cc00c57309a972d4fafee"
server_key_version: "encryptionkey@0"
I20260809 03:35:11.951655 6174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:11.969076 6174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:11.972343 6174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:11.972604 6174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:11.980154 6174 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:36863
I20260809 03:35:11.980165 6305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:36863 every 8 connection(s)
I20260809 03:35:11.981263 6174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:11.984220 6310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:11.990337 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 6174
I20260809 03:35:11.990504 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:11.990849 1921 external_mini_cluster.cc:1428] Setting key d556f792f6813aa038b2cbc374021d10
I20260809 03:35:11.991920 6310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053: Bootstrap starting.
I20260809 03:35:11.995342 6310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:11.996621 6310 log.cc:824] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:11.998747 6310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053: No bootstrap required, opened a new log
Aug 09 03:35:12 dist-test-slave-jq4t krb5kdc[6155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:12.004352 6310 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff82f9769a4944b0ae6ce44963e2b053" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 36863 } }
I20260809 03:35:12.004745 6310 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:12.004808 6310 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff82f9769a4944b0ae6ce44963e2b053, State: Initialized, Role: FOLLOWER
I20260809 03:35:12.005455 6310 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff82f9769a4944b0ae6ce44963e2b053" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 36863 } }
I20260809 03:35:12.005649 6310 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:12.005725 6310 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:12.005826 6310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:12.007043 6310 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff82f9769a4944b0ae6ce44963e2b053" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 36863 } }
I20260809 03:35:12.007452 6310 leader_election.cc:302] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff82f9769a4944b0ae6ce44963e2b053; no voters:
I20260809 03:35:12.007869 6310 leader_election.cc:288] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:12.008119 6316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:12.008407 6316 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [term 1 LEADER]: Becoming Leader. State: Replica: ff82f9769a4944b0ae6ce44963e2b053, State: Running, Role: LEADER
I20260809 03:35:12.008911 6316 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff82f9769a4944b0ae6ce44963e2b053" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 36863 } }
I20260809 03:35:12.009452 6310 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:12.010596 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:11.993195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44620 (local address 127.1.224.126:36863)
0809 03:35:11.993799 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:11.993829 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:11.993868 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:35:11.994586 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:11.994605 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:11.995053 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:11.996455 (+ 1402us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:11.996984 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:11.998031 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:11.999387 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.000095 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.000731 (+ 636us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:12.003778 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:12.003785 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:12.003806 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:12.003842 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:12.006293 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:12.006915 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:12.006925 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:12.006933 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:12.007105 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:12.007464 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:12.007470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:12.007472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:12.008095 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:12.009075 (+ 980us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:12.009233 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 03:35:12.009564 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":184,"threads_started":1}
I20260809 03:35:12.011312 6316 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff82f9769a4944b0ae6ce44963e2b053. Latest consensus state: current_term: 1 leader_uuid: "ff82f9769a4944b0ae6ce44963e2b053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff82f9769a4944b0ae6ce44963e2b053" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 36863 } } }
I20260809 03:35:12.011536 6316 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:12.011791 6317 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff82f9769a4944b0ae6ce44963e2b053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff82f9769a4944b0ae6ce44963e2b053" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 36863 } } }
I20260809 03:35:12.011890 6317 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053 [sys.catalog]: This master's current role is: LEADER
W20260809 03:35:12.018146 6350 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:35:12.018244 6350 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:35:12.018306 6340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:12.019290 6340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:12.025579 6340 catalog_manager.cc:1347] Generated new cluster ID: 19505fd720c24f76ac499881bc22193a
I20260809 03:35:12.025651 6340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:12.038801 6340 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:12.040310 6340 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:12.049868 6340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff82f9769a4944b0ae6ce44963e2b053: Generated new TSK 0
I20260809 03:35:12.051016 6340 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:35:12.091648 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:36863
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:12.212992 6362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:12.213343 6362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:12.213445 6362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:12.217625 6362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:12.217710 6362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:12.217795 6362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:12.223248 6362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:36863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:12.224428 6362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:12.225533 6362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:12.235790 6369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:12.235780 6371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:12.235791 6375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:12.235953 6362 server_base.cc:1034] running on GCE node
I20260809 03:35:12.236644 6362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:12.237311 6362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:12.238520 6362 hybrid_clock.cc:648] HybridClock initialized: now 1786246512238504 us; error 31 us; skew 500 ppm
Aug 09 03:35:12 dist-test-slave-jq4t krb5kdc[6155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246512, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:12.241467 6362 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:35:12.242409 6362 webserver.cc:460] Webserver started at http://127.1.224.65:35309/ using document root <none> and password file <none>
I20260809 03:35:12.243024 6362 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:12.243121 6362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:12.243366 6362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:12.245615 6362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "7f310a15f2364572b7a14fcf6a4e7d37"
format_stamp: "Formatted at 2026-08-09 03:35:12 on dist-test-slave-jq4t"
server_key: "e0469e8bafcd1acc6aacf9a854ce8e74"
server_key_iv: "1ee1d2eb693a963d21f7b88c34791c01"
server_key_version: "encryptionkey@0"
I20260809 03:35:12.246235 6362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "7f310a15f2364572b7a14fcf6a4e7d37"
format_stamp: "Formatted at 2026-08-09 03:35:12 on dist-test-slave-jq4t"
server_key: "e0469e8bafcd1acc6aacf9a854ce8e74"
server_key_iv: "1ee1d2eb693a963d21f7b88c34791c01"
server_key_version: "encryptionkey@0"
I20260809 03:35:12.250360 6362 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 03:35:12.253028 6383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:12.254240 6362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:12.254390 6362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "7f310a15f2364572b7a14fcf6a4e7d37"
format_stamp: "Formatted at 2026-08-09 03:35:12 on dist-test-slave-jq4t"
server_key: "e0469e8bafcd1acc6aacf9a854ce8e74"
server_key_iv: "1ee1d2eb693a963d21f7b88c34791c01"
server_key_version: "encryptionkey@0"
I20260809 03:35:12.254518 6362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:12.268258 6362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:12.271306 6362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:12.271522 6362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:12.272146 6362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:12.273247 6362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:12.273308 6362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:12.273357 6362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:12.273381 6362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:12.282342 6362 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:39185
I20260809 03:35:12.282371 6592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:39185 every 8 connection(s)
I20260809 03:35:12.283267 6362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:12.289858 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 6362
I20260809 03:35:12.289960 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:12.290241 1921 external_mini_cluster.cc:1428] Setting key ca6cb4a185e730e64086d3827ee4a45e
Aug 09 03:35:12 dist-test-slave-jq4t krb5kdc[6155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246512, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:12.298710 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:12.285221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:38363 (local address 127.1.224.126:36863)
0809 03:35:12.285367 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:12.285371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:12.286223 (+ 852us) server_negotiation.cc:407] Connection header received
0809 03:35:12.287552 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:12.287555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:12.287605 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:12.287682 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:12.288962 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.289512 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.290951 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.291240 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.291331 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:12.294480 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:12.294483 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:12.294486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:12.294514 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:12.296036 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:12.296848 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:12.296854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:12.296856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:12.296991 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:12.297424 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:12.297426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:12.297427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:12.297620 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:12.297696 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:12.298462 (+ 766us) server_negotiation.cc:299] Negotiation successful
0809 03:35:12.298581 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:35:12.300069 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:12.285611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:36863 (local address 127.1.224.65:38363)
0809 03:35:12.286066 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:12.286109 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:12.287351 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:12.287855 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:12.287861 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:12.288244 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:12.288818 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:12.288832 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.289651 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.289653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:12.290799 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:12.290813 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.291413 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.291417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:12.291633 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:12.292425 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:12.292444 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:12.294253 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:12.296167 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:12.296177 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:12.296197 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:12.296717 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:12.297102 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:12.297108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:12.297113 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:12.297312 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:12.297740 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:12.297752 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:12.298159 (+ 407us) client_negotiation.cc:769] Sending connection context
0809 03:35:12.298414 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 03:35:12.298847 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":117,"threads_started":1}
I20260809 03:35:12.301483 6593 heartbeater.cc:347] Connected to a master server at 127.1.224.126:36863
I20260809 03:35:12.301751 6593 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:35:12.302294 6593 heartbeater.cc:511] Master 127.1.224.126:36863 requested a full tablet report, sending...
I20260809 03:35:12.304095 6243 ts_manager.cc:194] Registered new tserver with Master: 7f310a15f2364572b7a14fcf6a4e7d37 (127.1.224.65:39185)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:35:12.305740 6243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:38363
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:35:12.324259 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:36863
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:12.444182 6605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:12.444470 6605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:12.444634 6605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:12.448791 6605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:12.448853 6605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:12.448913 6605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:12.454483 6605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:36863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:12.455717 6605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:12.456884 6605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:35:12.467350 6605 server_base.cc:1034] running on GCE node
W20260809 03:35:12.467298 6616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:12.467330 6612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:12.467298 6619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:12.467926 6605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:12.468664 6605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:12.469851 6605 hybrid_clock.cc:648] HybridClock initialized: now 1786246512469838 us; error 41 us; skew 500 ppm
Aug 09 03:35:12 dist-test-slave-jq4t krb5kdc[6155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246512, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:12.472786 6605 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:35:12.473747 6605 webserver.cc:460] Webserver started at http://127.1.224.66:38349/ using document root <none> and password file <none>
I20260809 03:35:12.474289 6605 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:12.474347 6605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:12.474532 6605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:12.476656 6605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "2753847890644776baceaa9bc37a4f68"
format_stamp: "Formatted at 2026-08-09 03:35:12 on dist-test-slave-jq4t"
server_key: "f84c7cb44db332fc3c66746aa35d2a6c"
server_key_iv: "1d90bc7b3cdb0bffe40fea297c2aa5c5"
server_key_version: "encryptionkey@0"
I20260809 03:35:12.477185 6605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "2753847890644776baceaa9bc37a4f68"
format_stamp: "Formatted at 2026-08-09 03:35:12 on dist-test-slave-jq4t"
server_key: "f84c7cb44db332fc3c66746aa35d2a6c"
server_key_iv: "1d90bc7b3cdb0bffe40fea297c2aa5c5"
server_key_version: "encryptionkey@0"
I20260809 03:35:12.480945 6605 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 03:35:12.483291 6630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:12.484579 6605 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:12.484673 6605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "2753847890644776baceaa9bc37a4f68"
format_stamp: "Formatted at 2026-08-09 03:35:12 on dist-test-slave-jq4t"
server_key: "f84c7cb44db332fc3c66746aa35d2a6c"
server_key_iv: "1d90bc7b3cdb0bffe40fea297c2aa5c5"
server_key_version: "encryptionkey@0"
I20260809 03:35:12.484756 6605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:12.498726 6605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:12.501930 6605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:12.502095 6605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:12.502665 6605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:12.503583 6605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:12.503634 6605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:12.503676 6605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:12.503696 6605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:12.513705 6605 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:46481
I20260809 03:35:12.513728 6825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:46481 every 8 connection(s)
I20260809 03:35:12.514746 6605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:12.522184 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 6605
I20260809 03:35:12.522327 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:12.522701 1921 external_mini_cluster.cc:1428] Setting key d266569e679918d6164c5e4089770046
Aug 09 03:35:12 dist-test-slave-jq4t krb5kdc[6155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246512, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:12.530318 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:12.516690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:45989 (local address 127.1.224.126:36863)
0809 03:35:12.516847 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:12.516852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:12.517747 (+ 895us) server_negotiation.cc:407] Connection header received
0809 03:35:12.519103 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:12.519107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:12.519158 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:12.519237 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:12.520571 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.521119 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.522498 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.522844 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.523015 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:12.526139 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:12.526146 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:12.526152 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:12.526189 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:12.527746 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:12.528413 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:12.528416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:12.528417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:12.528560 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:12.528986 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:12.528989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:12.528991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:12.529218 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:12.529301 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:12.530074 (+ 773us) server_negotiation.cc:299] Negotiation successful
0809 03:35:12.530190 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:35:12.531747 6830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:12.517148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:36863 (local address 127.1.224.66:45989)
0809 03:35:12.517593 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:12.517637 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:12.518880 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:12.519444 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:12.519451 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:12.519822 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:12.520368 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:12.520383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.521275 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.521278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:12.522263 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:12.522274 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.523050 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.523054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:12.523273 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:12.523999 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:12.524017 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:12.525947 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:12.527883 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:12.527893 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:12.527913 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:12.528305 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:12.528656 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:12.528661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:12.528666 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:12.528870 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:12.529338 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:12.529348 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:12.529764 (+ 416us) client_negotiation.cc:769] Sending connection context
0809 03:35:12.530018 (+ 254us) client_negotiation.cc:240] Negotiation successful
0809 03:35:12.530433 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":105,"threads_started":1}
I20260809 03:35:12.533138 6826 heartbeater.cc:347] Connected to a master server at 127.1.224.126:36863
I20260809 03:35:12.533411 6826 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:12.533957 6826 heartbeater.cc:511] Master 127.1.224.126:36863 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:35:12.535137 6243 ts_manager.cc:194] Registered new tserver with Master: 2753847890644776baceaa9bc37a4f68 (127.1.224.66:46481)
I20260809 03:35:12.535756 6243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:45989
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:35:12.555202 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:36863
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:12.676652 6833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:12.676975 6833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:12.677053 6833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:12.681116 6833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:12.681207 6833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:12.681298 6833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:12.686826 6833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:36863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:12.688063 6833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:12.689275 6833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:12.699872 6848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:12.699862 6842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:12.699855 6843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:12.700075 6833 server_base.cc:1034] running on GCE node
I20260809 03:35:12.700621 6833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:12.701473 6833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:12.702661 6833 hybrid_clock.cc:648] HybridClock initialized: now 1786246512702632 us; error 49 us; skew 500 ppm
Aug 09 03:35:12 dist-test-slave-jq4t krb5kdc[6155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246512, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:12.705762 6833 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:35:12.706691 6833 webserver.cc:460] Webserver started at http://127.1.224.67:38139/ using document root <none> and password file <none>
I20260809 03:35:12.707295 6833 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:12.707389 6833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:12.707638 6833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:12.709870 6833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "6769303094854ad2bcd3f86590b0c8b4"
format_stamp: "Formatted at 2026-08-09 03:35:12 on dist-test-slave-jq4t"
server_key: "1b0d26fac0c5e7365dcfa4214e428987"
server_key_iv: "e07a0acd71b9aa07a971b5b1ccefc4d5"
server_key_version: "encryptionkey@0"
I20260809 03:35:12.710458 6833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "6769303094854ad2bcd3f86590b0c8b4"
format_stamp: "Formatted at 2026-08-09 03:35:12 on dist-test-slave-jq4t"
server_key: "1b0d26fac0c5e7365dcfa4214e428987"
server_key_iv: "e07a0acd71b9aa07a971b5b1ccefc4d5"
server_key_version: "encryptionkey@0"
I20260809 03:35:12.714632 6833 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:12.717146 6858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:12.718369 6833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:12.718516 6833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "6769303094854ad2bcd3f86590b0c8b4"
format_stamp: "Formatted at 2026-08-09 03:35:12 on dist-test-slave-jq4t"
server_key: "1b0d26fac0c5e7365dcfa4214e428987"
server_key_iv: "e07a0acd71b9aa07a971b5b1ccefc4d5"
server_key_version: "encryptionkey@0"
I20260809 03:35:12.718632 6833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:12.735489 6833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:12.738744 6833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:12.738965 6833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:12.739614 6833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:12.740579 6833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:12.740646 6833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:12.740720 6833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:12.740772 6833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:12.752125 6833 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:41389
I20260809 03:35:12.752135 7055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:41389 every 8 connection(s)
I20260809 03:35:12.753212 6833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:12.763161 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 6833
I20260809 03:35:12.763316 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:12.763661 1921 external_mini_cluster.cc:1428] Setting key 31270cd0eaefcd1c77e58e0b6468a3ad
Aug 09 03:35:12 dist-test-slave-jq4t krb5kdc[6155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246512, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:12.768150 6313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:12.755292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:39729 (local address 127.1.224.126:36863)
0809 03:35:12.755447 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:12.755451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:12.756347 (+ 896us) server_negotiation.cc:407] Connection header received
0809 03:35:12.757864 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:12.757868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:12.757946 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:12.758052 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:12.759411 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.759980 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.761259 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.761496 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.761612 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:12.764546 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:12.764550 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:12.764553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:12.764581 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:12.766247 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:12.766742 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:12.766744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:12.766746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:12.766828 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:12.767143 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:12.767145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:12.767146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:12.767328 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:12.767404 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:12.767917 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 03:35:12.768024 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:35:12.769130 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:12.755718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:36863 (local address 127.1.224.67:39729)
0809 03:35:12.756188 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:12.756232 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:12.757636 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:12.758284 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:12.758295 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:12.758699 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:12.759229 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:12.759243 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.760116 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.760120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:12.761121 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:12.761132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:12.761663 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:12.761667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:12.761854 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:12.762488 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:12.762505 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:12.764289 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:12.766361 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:12.766366 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:12.766380 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:12.766652 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:12.766913 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:12.766915 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:12.766917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:12.767060 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:12.767426 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:12.767431 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:12.767693 (+ 262us) client_negotiation.cc:769] Sending connection context
0809 03:35:12.767864 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 03:35:12.768139 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":125,"threads_started":1}
I20260809 03:35:12.770409 7057 heartbeater.cc:347] Connected to a master server at 127.1.224.126:36863
I20260809 03:35:12.770671 7057 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:12.771230 7057 heartbeater.cc:511] Master 127.1.224.126:36863 requested a full tablet report, sending...
I20260809 03:35:12.772396 6243 ts_manager.cc:194] Registered new tserver with Master: 6769303094854ad2bcd3f86590b0c8b4 (127.1.224.67:41389)
I20260809 03:35:12.773077 6243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:39729
I20260809 03:35:12.778352 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 03:35:12.794849 6313 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:12.793329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44622 (local address 127.1.224.126:36863)
0809 03:35:12.793538 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:12.793542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:12.793610 (+ 68us) server_negotiation.cc:407] Connection header received
0809 03:35:12.793734 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:12.793737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:12.793784 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:12.793891 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:12.794689 (+ 798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44622: BlockingRecv error: recv got EOF from 127.0.0.1:44622 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 03:35:12.795925 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 6362
I20260809 03:35:12.813560 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 6605
I20260809 03:35:12.833585 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 6833
I20260809 03:35:12.843938 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 6174
2026-08-09T03:35:12Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2439 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:12 dist-test-slave-jq4t krb5kdc[7077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:12 dist-test-slave-jq4t krb5kdc[7077](info): set up 2 sockets
Aug 09 03:35:12 dist-test-slave-jq4t krb5kdc[7077](info): commencing operation
krb5kdc: starting...
W20260809 03:35:14.094621 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:35:14 dist-test-slave-jq4t krb5kdc[7077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:35:14.232789 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:34835
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:34835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:14.353822 7096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:14.354183 7096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:14.354269 7096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:14.358496 7096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:14.358584 7096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:14.358620 7096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:14.358675 7096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:14.358727 7096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:14.364295 7096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:34835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:34835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:14.365521 7096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:14.366605 7096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:14.375855 7106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:14.375811 7105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:14.375821 7111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:14.376072 7096 server_base.cc:1034] running on GCE node
I20260809 03:35:14.376905 7096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:14.378185 7096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:14.379392 7096 hybrid_clock.cc:648] HybridClock initialized: now 1786246514379377 us; error 33 us; skew 500 ppm
Aug 09 03:35:14 dist-test-slave-jq4t krb5kdc[7077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246514, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:14.387183 7096 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:35:14.388669 7096 webserver.cc:460] Webserver started at http://127.1.224.126:44719/ using document root <none> and password file <none>
I20260809 03:35:14.389427 7096 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:14.389537 7096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:14.389798 7096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:14.392211 7096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "746681037b0e429aaf7a51558b667d7d"
format_stamp: "Formatted at 2026-08-09 03:35:14 on dist-test-slave-jq4t"
server_key: "b3438fa2cc29e056749b3a5d847efc79"
server_key_iv: "5ff5f4322c52769c2d504d83f8cfaadc"
server_key_version: "encryptionkey@0"
I20260809 03:35:14.392922 7096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "746681037b0e429aaf7a51558b667d7d"
format_stamp: "Formatted at 2026-08-09 03:35:14 on dist-test-slave-jq4t"
server_key: "b3438fa2cc29e056749b3a5d847efc79"
server_key_iv: "5ff5f4322c52769c2d504d83f8cfaadc"
server_key_version: "encryptionkey@0"
I20260809 03:35:14.397087 7096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:35:14.399685 7123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:14.401139 7096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:14.401297 7096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "746681037b0e429aaf7a51558b667d7d"
format_stamp: "Formatted at 2026-08-09 03:35:14 on dist-test-slave-jq4t"
server_key: "b3438fa2cc29e056749b3a5d847efc79"
server_key_iv: "5ff5f4322c52769c2d504d83f8cfaadc"
server_key_version: "encryptionkey@0"
I20260809 03:35:14.401423 7096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:14.416592 7096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:14.420022 7096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:14.420255 7096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:14.427956 7096 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:34835
I20260809 03:35:14.428306 7210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:34835 every 8 connection(s)
I20260809 03:35:14.429093 7096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:14.431033 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 7096
I20260809 03:35:14.431177 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:14.431457 1921 external_mini_cluster.cc:1428] Setting key 9969a588e603ca7c5eb11077ae54d653
I20260809 03:35:14.432220 7211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:14.439266 7211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d: Bootstrap starting.
Aug 09 03:35:14 dist-test-slave-jq4t krb5kdc[7077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:14.442121 7211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:14.443146 7211 log.cc:824] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:14.445719 7211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d: No bootstrap required, opened a new log
I20260809 03:35:14.447067 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:14.432895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53618 (local address 127.1.224.126:34835)
0809 03:35:14.433352 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:14.433369 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:14.433412 (+ 43us) server_negotiation.cc:407] Connection header received
0809 03:35:14.434097 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:14.434115 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:14.434452 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:14.434758 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:14.435773 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:14.436936 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:14.438185 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:14.438670 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:14.438790 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:14.441720 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:14.441726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:14.441745 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:14.441779 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:14.443806 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:14.444302 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:14.444309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:14.444315 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:14.444452 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:14.444764 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:14.444768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:14.444769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:14.445321 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:14.445528 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:14.445740 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 03:35:14.446089 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":158,"threads_started":1}
I20260809 03:35:14.449457 7211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "746681037b0e429aaf7a51558b667d7d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 34835 } }
I20260809 03:35:14.449785 7211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:14.449852 7211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 746681037b0e429aaf7a51558b667d7d, State: Initialized, Role: FOLLOWER
I20260809 03:35:14.450341 7211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "746681037b0e429aaf7a51558b667d7d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 34835 } }
I20260809 03:35:14.450522 7211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:14.450592 7211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:14.450698 7211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:14.450901 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:35:14.451778 7211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "746681037b0e429aaf7a51558b667d7d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 34835 } }
I20260809 03:35:14.452145 7211 leader_election.cc:302] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 746681037b0e429aaf7a51558b667d7d; no voters:
I20260809 03:35:14.452548 7211 leader_election.cc:288] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:14.452644 7220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:14.452896 7220 raft_consensus.cc:686] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [term 1 LEADER]: Becoming Leader. State: Replica: 746681037b0e429aaf7a51558b667d7d, State: Running, Role: LEADER
I20260809 03:35:14.453253 7220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "746681037b0e429aaf7a51558b667d7d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 34835 } }
I20260809 03:35:14.453895 7211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:14.455156 7221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "746681037b0e429aaf7a51558b667d7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "746681037b0e429aaf7a51558b667d7d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 34835 } } }
I20260809 03:35:14.455309 7221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:14.455667 7222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 746681037b0e429aaf7a51558b667d7d. Latest consensus state: current_term: 1 leader_uuid: "746681037b0e429aaf7a51558b667d7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "746681037b0e429aaf7a51558b667d7d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 34835 } } }
I20260809 03:35:14.455780 7222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d [sys.catalog]: This master's current role is: LEADER
W20260809 03:35:14.459174 7243 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:35:14.459282 7243 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:35:14.459372 7233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:14.460186 7233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:14.467115 7233 catalog_manager.cc:1347] Generated new cluster ID: 9e7bd51a7f344e07a32966cb0333b72c
I20260809 03:35:14.467191 7233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:14.472473 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:14.467558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53626 (local address 127.1.224.126:34835)
0809 03:35:14.467691 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:14.467696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:14.467885 (+ 189us) server_negotiation.cc:407] Connection header received
0809 03:35:14.468018 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:14.468022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:14.468075 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:14.468171 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:14.468773 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:14.469323 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:14.470180 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:14.470461 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:14.470574 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:14.471514 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:14.471518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:14.471521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:14.471972 (+ 451us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:14.472003 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:14.472020 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:14.472122 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:14.472244 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 03:35:14.472312 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":41}
I20260809 03:35:14.478636 7233 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:14.479969 7233 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:14.493820 7233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 746681037b0e429aaf7a51558b667d7d: Generated new TSK 0
I20260809 03:35:14.494848 7233 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:35:14.503464 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 7096
2026-08-09T03:35:14Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1659 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:14 dist-test-slave-jq4t krb5kdc[7266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:14 dist-test-slave-jq4t krb5kdc[7266](info): set up 2 sockets
Aug 09 03:35:14 dist-test-slave-jq4t krb5kdc[7266](info): commencing operation
krb5kdc: starting...
W20260809 03:35:15.775787 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:35:15 dist-test-slave-jq4t krb5kdc[7266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:35:15.911549 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:45263
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:45263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:16.033306 7287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:16.033628 7287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:16.033674 7287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:16.037899 7287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:16.037972 7287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:16.037993 7287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:16.038012 7287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:16.038028 7287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:16.044140 7287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:45263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:45263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:16.045343 7287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:16.046443 7287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:16.055578 7298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:16.055670 7295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:16.055585 7296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:16.055816 7287 server_base.cc:1034] running on GCE node
I20260809 03:35:16.056650 7287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:16.057832 7287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:16.059036 7287 hybrid_clock.cc:648] HybridClock initialized: now 1786246516059014 us; error 43 us; skew 500 ppm
Aug 09 03:35:16 dist-test-slave-jq4t krb5kdc[7266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:16.062053 7287 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:35:16.063045 7287 webserver.cc:460] Webserver started at http://127.1.224.126:46847/ using document root <none> and password file <none>
I20260809 03:35:16.063674 7287 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:16.063773 7287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:16.064035 7287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:16.066354 7287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "2d01cd52ba694894abb655e51f53e3d8"
format_stamp: "Formatted at 2026-08-09 03:35:16 on dist-test-slave-jq4t"
server_key: "1494b7ddc7c02d10453eb5b2f7958d0f"
server_key_iv: "e58b44ec4953782f60ee4b6376127410"
server_key_version: "encryptionkey@0"
I20260809 03:35:16.066979 7287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "2d01cd52ba694894abb655e51f53e3d8"
format_stamp: "Formatted at 2026-08-09 03:35:16 on dist-test-slave-jq4t"
server_key: "1494b7ddc7c02d10453eb5b2f7958d0f"
server_key_iv: "e58b44ec4953782f60ee4b6376127410"
server_key_version: "encryptionkey@0"
I20260809 03:35:16.071170 7287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260809 03:35:16.073807 7311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:16.075063 7287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 03:35:16.075218 7287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "2d01cd52ba694894abb655e51f53e3d8"
format_stamp: "Formatted at 2026-08-09 03:35:16 on dist-test-slave-jq4t"
server_key: "1494b7ddc7c02d10453eb5b2f7958d0f"
server_key_iv: "e58b44ec4953782f60ee4b6376127410"
server_key_version: "encryptionkey@0"
I20260809 03:35:16.075340 7287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:16.091673 7287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:16.094884 7287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:16.095125 7287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:16.102756 7287 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:45263
I20260809 03:35:16.103097 7395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:45263 every 8 connection(s)
I20260809 03:35:16.103916 7287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:16.107060 7396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:16.109841 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 7287
I20260809 03:35:16.109998 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:16.110314 1921 external_mini_cluster.cc:1428] Setting key 3ebe9df7edea073a6f149f98ddbfa725
I20260809 03:35:16.114174 7396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8: Bootstrap starting.
I20260809 03:35:16.117067 7396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:16.118022 7396 log.cc:824] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:16.120443 7396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8: No bootstrap required, opened a new log
Aug 09 03:35:16 dist-test-slave-jq4t krb5kdc[7266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:16.124274 7396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d01cd52ba694894abb655e51f53e3d8" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45263 } }
I20260809 03:35:16.124588 7396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:16.124670 7396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d01cd52ba694894abb655e51f53e3d8, State: Initialized, Role: FOLLOWER
I20260809 03:35:16.125190 7396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d01cd52ba694894abb655e51f53e3d8" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45263 } }
I20260809 03:35:16.125324 7396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:16.125417 7396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:16.125535 7396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:16.126566 7396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d01cd52ba694894abb655e51f53e3d8" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45263 } }
I20260809 03:35:16.126684 7402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:16.111962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58760 (local address 127.1.224.126:45263)
0809 03:35:16.112415 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:16.112438 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:16.112516 (+ 78us) server_negotiation.cc:407] Connection header received
0809 03:35:16.113382 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:16.113411 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:16.113836 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:16.114242 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:16.115042 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.116631 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.117581 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.118306 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.118493 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:16.121479 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:16.121484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:16.121501 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:16.121530 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:16.123379 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:16.123876 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:16.123882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:16.123887 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:16.124025 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:16.124346 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:16.124350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:16.124351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:16.124970 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:16.125200 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:16.125569 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 03:35:16.125912 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":105,"threads_started":1}
I20260809 03:35:16.127003 7396 leader_election.cc:302] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d01cd52ba694894abb655e51f53e3d8; no voters:
I20260809 03:35:16.127398 7396 leader_election.cc:288] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:16.127516 7404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:16.127743 7404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [term 1 LEADER]: Becoming Leader. State: Replica: 2d01cd52ba694894abb655e51f53e3d8, State: Running, Role: LEADER
I20260809 03:35:16.128060 7404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d01cd52ba694894abb655e51f53e3d8" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45263 } }
I20260809 03:35:16.128752 7396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:16.129810 7408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d01cd52ba694894abb655e51f53e3d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d01cd52ba694894abb655e51f53e3d8" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45263 } } }
I20260809 03:35:16.129953 7408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:16.130309 7409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d01cd52ba694894abb655e51f53e3d8. Latest consensus state: current_term: 1 leader_uuid: "2d01cd52ba694894abb655e51f53e3d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d01cd52ba694894abb655e51f53e3d8" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45263 } } }
I20260809 03:35:16.130401 7422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:16.130429 7409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:16.133492 7422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:16.140254 7422 catalog_manager.cc:1347] Generated new cluster ID: cea9355141a34b41977986a5915b17d3
I20260809 03:35:16.140359 7422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:16.155104 7422 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:16.156457 7422 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:16.162067 7422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d01cd52ba694894abb655e51f53e3d8: Generated new TSK 0
I20260809 03:35:16.162880 7422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:35:16.197456 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45263
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:16.319985 7449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:16.320319 7449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:16.320365 7449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:16.324635 7449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:16.324707 7449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:16.324772 7449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:16.330647 7449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:16.331920 7449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:16.333061 7449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:16.343648 7454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:16.343641 7457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:16.343740 7460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:16.344033 7449 server_base.cc:1034] running on GCE node
I20260809 03:35:16.344578 7449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:16.345283 7449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:16.346447 7449 hybrid_clock.cc:648] HybridClock initialized: now 1786246516346414 us; error 46 us; skew 500 ppm
Aug 09 03:35:16 dist-test-slave-jq4t krb5kdc[7266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:16.349661 7449 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:35:16.350672 7449 webserver.cc:460] Webserver started at http://127.1.224.65:41273/ using document root <none> and password file <none>
I20260809 03:35:16.351294 7449 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:16.351359 7449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:16.351640 7449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:16.353856 7449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "3f9f71e472ed452084d85eaa69bc3b19"
format_stamp: "Formatted at 2026-08-09 03:35:16 on dist-test-slave-jq4t"
server_key: "4f18d3c5172b78d6113b674661f5b4e2"
server_key_iv: "d0845bbd6608fbea4eb7555bbeaf2ba3"
server_key_version: "encryptionkey@0"
I20260809 03:35:16.354462 7449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "3f9f71e472ed452084d85eaa69bc3b19"
format_stamp: "Formatted at 2026-08-09 03:35:16 on dist-test-slave-jq4t"
server_key: "4f18d3c5172b78d6113b674661f5b4e2"
server_key_iv: "d0845bbd6608fbea4eb7555bbeaf2ba3"
server_key_version: "encryptionkey@0"
I20260809 03:35:16.358390 7449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:16.360875 7469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:16.362366 7449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:16.362521 7449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "3f9f71e472ed452084d85eaa69bc3b19"
format_stamp: "Formatted at 2026-08-09 03:35:16 on dist-test-slave-jq4t"
server_key: "4f18d3c5172b78d6113b674661f5b4e2"
server_key_iv: "d0845bbd6608fbea4eb7555bbeaf2ba3"
server_key_version: "encryptionkey@0"
I20260809 03:35:16.362637 7449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:16.376236 7449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:16.379403 7449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:16.379611 7449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:16.380290 7449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:16.381261 7449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:16.381311 7449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:16.381382 7449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:16.381413 7449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:16.390748 7449 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:42703
I20260809 03:35:16.390789 7643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:42703 every 8 connection(s)
I20260809 03:35:16.392087 7449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:16.395474 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 7449
I20260809 03:35:16.395627 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:16.395920 1921 external_mini_cluster.cc:1428] Setting key 6532f9ef3d0152fc3b114d6c4bdf9ec8
Aug 09 03:35:16 dist-test-slave-jq4t krb5kdc[7266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:35:16.408880 7402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:16.395429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:48659 (local address 127.1.224.126:45263)
0809 03:35:16.395656 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:16.395661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:16.396701 (+ 1040us) server_negotiation.cc:407] Connection header received
0809 03:35:16.398256 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:16.398259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:16.398315 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:16.398399 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:16.399654 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.400222 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.401493 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.401745 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.401864 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:16.404846 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:16.404852 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:16.404857 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:16.404892 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:16.406643 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:16.407219 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:16.407222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:16.407224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:16.407307 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:16.407635 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:16.407637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:16.407638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:16.407910 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:16.407990 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:16.408589 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 03:35:16.408708 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:35:16.409895 7649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:16.395907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:45263 (local address 127.1.224.65:48659)
0809 03:35:16.396526 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:16.396599 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:16.398030 (+ 1431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:16.398597 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:16.398604 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:16.398979 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:16.399483 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:16.399498 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.400363 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.400367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:16.401349 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:16.401360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.401913 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.401917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:16.402105 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:16.402778 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:16.402795 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:16.404653 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:16.406795 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:16.406801 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:16.406814 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:16.407099 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:16.407395 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:16.407397 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:16.407399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:16.407540 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:16.408016 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:16.408022 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:16.408301 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 03:35:16.408466 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 03:35:16.408869 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":172,"threads_started":1}
I20260809 03:35:16.411314 7644 heartbeater.cc:347] Connected to a master server at 127.1.224.126:45263
I20260809 03:35:16.411624 7644 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:16.412205 7644 heartbeater.cc:511] Master 127.1.224.126:45263 requested a full tablet report, sending...
I20260809 03:35:16.414156 7334 ts_manager.cc:194] Registered new tserver with Master: 3f9f71e472ed452084d85eaa69bc3b19 (127.1.224.65:42703)
I20260809 03:35:16.415995 7334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:48659
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:35:16.427945 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45263
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:16.547003 7656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:16.547269 7656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:16.547313 7656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:16.551334 7656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:16.551395 7656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:16.551455 7656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:16.556772 7656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:16.557823 7656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:16.558835 7656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:16.569339 7664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:16.569344 7668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:16.569515 7656 server_base.cc:1034] running on GCE node
W20260809 03:35:16.569557 7666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:16.570142 7656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:16.570765 7656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:16.571935 7656 hybrid_clock.cc:648] HybridClock initialized: now 1786246516571921 us; error 31 us; skew 500 ppm
Aug 09 03:35:16 dist-test-slave-jq4t krb5kdc[7266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:16.574887 7656 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:35:16.575932 7656 webserver.cc:460] Webserver started at http://127.1.224.66:35243/ using document root <none> and password file <none>
I20260809 03:35:16.576627 7656 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:16.576726 7656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:16.576967 7656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:16.579237 7656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "57ce91debd1143c899b703014b862b14"
format_stamp: "Formatted at 2026-08-09 03:35:16 on dist-test-slave-jq4t"
server_key: "2fe5da7704695a81c72c2a2aa02adb4e"
server_key_iv: "b5a08b444752866f2aad6d4652fd95a1"
server_key_version: "encryptionkey@0"
I20260809 03:35:16.579913 7656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "57ce91debd1143c899b703014b862b14"
format_stamp: "Formatted at 2026-08-09 03:35:16 on dist-test-slave-jq4t"
server_key: "2fe5da7704695a81c72c2a2aa02adb4e"
server_key_iv: "b5a08b444752866f2aad6d4652fd95a1"
server_key_version: "encryptionkey@0"
I20260809 03:35:16.583827 7656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260809 03:35:16.586344 7678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:16.587604 7656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 03:35:16.587761 7656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "57ce91debd1143c899b703014b862b14"
format_stamp: "Formatted at 2026-08-09 03:35:16 on dist-test-slave-jq4t"
server_key: "2fe5da7704695a81c72c2a2aa02adb4e"
server_key_iv: "b5a08b444752866f2aad6d4652fd95a1"
server_key_version: "encryptionkey@0"
I20260809 03:35:16.587879 7656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:16.598384 7656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:16.601694 7656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:16.601919 7656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:16.602556 7656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:16.603463 7656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:16.603541 7656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:16.603621 7656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:16.603667 7656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:16.613570 7656 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:33019
I20260809 03:35:16.613596 7899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:33019 every 8 connection(s)
I20260809 03:35:16.614693 7656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:16.616066 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 7656
I20260809 03:35:16.616206 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:16.616562 1921 external_mini_cluster.cc:1428] Setting key 05cff05d2e4370abed0600008a00f164
Aug 09 03:35:16 dist-test-slave-jq4t krb5kdc[7266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
I20260809 03:35:16.631835 7402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:16.616984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:35911 (local address 127.1.224.126:45263)
0809 03:35:16.617172 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:16.617177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:16.618204 (+ 1027us) server_negotiation.cc:407] Connection header received
0809 03:35:16.619683 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:16.619690 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:16.619759 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:16.619856 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:16.621620 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.622201 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.624149 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.624386 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.624543 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:16.627594 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:16.627597 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:16.627599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:16.627624 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:16.629380 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:16.629958 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:16.629965 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:16.629970 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:16.630069 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:16.630518 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:16.630521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:16.630523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:16.630734 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:16.630871 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:16.631518 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 03:35:16.631656 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 03:35:16.632784 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:16.617426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:45263 (local address 127.1.224.66:35911)
0809 03:35:16.618021 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:16.618087 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:16.619448 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:16.620069 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:16.620077 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:16.620674 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:16.621444 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:16.621467 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.622363 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.622370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:16.624008 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:16.624023 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.624631 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.624639 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:16.624916 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:16.625662 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:16.625690 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:16.627399 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:16.629508 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:16.629514 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:16.629531 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:16.629839 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:16.630169 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:16.630172 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:16.630174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:16.630403 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:16.630872 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:16.630879 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:16.631243 (+ 364us) client_negotiation.cc:769] Sending connection context
0809 03:35:16.631464 (+ 221us) client_negotiation.cc:240] Negotiation successful
0809 03:35:16.631821 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":130,"threads_started":1}
I20260809 03:35:16.634178 7901 heartbeater.cc:347] Connected to a master server at 127.1.224.126:45263
I20260809 03:35:16.634486 7901 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:16.635064 7901 heartbeater.cc:511] Master 127.1.224.126:45263 requested a full tablet report, sending...
I20260809 03:35:16.636194 7334 ts_manager.cc:194] Registered new tserver with Master: 57ce91debd1143c899b703014b862b14 (127.1.224.66:33019)
I20260809 03:35:16.637147 7334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:35911
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:35:16.649767 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45263
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:16.770021 7911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:16.770315 7911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:16.770361 7911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:16.774564 7911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:16.774646 7911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:16.774709 7911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:16.780966 7911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:16.782137 7911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:16.783185 7911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:16.793367 7918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:16.793495 7911 server_base.cc:1034] running on GCE node
W20260809 03:35:16.793336 7920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:16.793675 7917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:16.794113 7911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:16.794795 7911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:16.795997 7911 hybrid_clock.cc:648] HybridClock initialized: now 1786246516795965 us; error 45 us; skew 500 ppm
Aug 09 03:35:16 dist-test-slave-jq4t krb5kdc[7266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:16.798889 7911 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:35:16.799819 7911 webserver.cc:460] Webserver started at http://127.1.224.67:45679/ using document root <none> and password file <none>
I20260809 03:35:16.800417 7911 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:16.800537 7911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:16.800786 7911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:16.802847 7911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "1dfa5e57f55140aeaa491fc6c857cb62"
format_stamp: "Formatted at 2026-08-09 03:35:16 on dist-test-slave-jq4t"
server_key: "67c4930c8378de853f7045df96b7b454"
server_key_iv: "a6ea869085450777894a707e34e1a437"
server_key_version: "encryptionkey@0"
I20260809 03:35:16.803436 7911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "1dfa5e57f55140aeaa491fc6c857cb62"
format_stamp: "Formatted at 2026-08-09 03:35:16 on dist-test-slave-jq4t"
server_key: "67c4930c8378de853f7045df96b7b454"
server_key_iv: "a6ea869085450777894a707e34e1a437"
server_key_version: "encryptionkey@0"
I20260809 03:35:16.807201 7911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:35:16.809620 7927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:16.810848 7911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:35:16.811004 7911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "1dfa5e57f55140aeaa491fc6c857cb62"
format_stamp: "Formatted at 2026-08-09 03:35:16 on dist-test-slave-jq4t"
server_key: "67c4930c8378de853f7045df96b7b454"
server_key_iv: "a6ea869085450777894a707e34e1a437"
server_key_version: "encryptionkey@0"
I20260809 03:35:16.811120 7911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:16.831489 7911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:16.834722 7911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:16.834956 7911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:16.835595 7911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:16.836469 7911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:16.836570 7911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:16.836645 7911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:16.836689 7911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:16.845531 7911 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:39947
I20260809 03:35:16.845561 8106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:39947 every 8 connection(s)
I20260809 03:35:16.846432 7911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:16.847513 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 7911
I20260809 03:35:16.847610 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:16.848012 1921 external_mini_cluster.cc:1428] Setting key 4deeb926a952f4af155a6ff5bc9d9e7e
Aug 09 03:35:16 dist-test-slave-jq4t krb5kdc[7266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:16.860442 7402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:16.848479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:47707 (local address 127.1.224.126:45263)
0809 03:35:16.848648 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:16.848653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:16.849769 (+ 1116us) server_negotiation.cc:407] Connection header received
0809 03:35:16.851297 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:16.851299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:16.851344 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:16.851414 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:16.852609 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.853149 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.854285 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.854494 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.854570 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:16.857002 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:16.857005 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:16.857007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:16.857032 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:16.858585 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:16.859053 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:16.859056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:16.859057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:16.859138 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:16.859445 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:16.859448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:16.859449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:16.859634 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:16.859707 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:16.860201 (+ 494us) server_negotiation.cc:299] Negotiation successful
0809 03:35:16.860309 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:35:16.861558 8112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:16.849007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:45263 (local address 127.1.224.67:47707)
0809 03:35:16.849579 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:16.849652 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:16.851107 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:16.851589 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:16.851595 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:16.851942 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:16.852436 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:16.852450 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.853265 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.853268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:16.854178 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:16.854187 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.854622 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.854625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:16.854765 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:16.855362 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:16.855377 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:16.856846 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:16.858687 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:16.858692 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:16.858704 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:16.858956 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:16.859216 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:16.859218 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:16.859220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:16.859355 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:16.859720 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:16.859725 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:16.859968 (+ 243us) client_negotiation.cc:769] Sending connection context
0809 03:35:16.860132 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 03:35:16.860529 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":148,"threads_started":1}
I20260809 03:35:16.862753 8110 heartbeater.cc:347] Connected to a master server at 127.1.224.126:45263
I20260809 03:35:16.862975 8110 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:16.863467 8110 heartbeater.cc:511] Master 127.1.224.126:45263 requested a full tablet report, sending...
I20260809 03:35:16.864665 7334 ts_manager.cc:194] Registered new tserver with Master: 1dfa5e57f55140aeaa491fc6c857cb62 (127.1.224.67:39947)
I20260809 03:35:16.865273 7334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:47707
I20260809 03:35:16.874752 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:35:16 dist-test-slave-jq4t krb5kdc[7266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246516, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 03:35:16 dist-test-slave-jq4t krb5kdc[7266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246516, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:16.903165 7402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:16.894255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58774 (local address 127.1.224.126:45263)
0809 03:35:16.894409 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:16.894415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:16.894589 (+ 174us) server_negotiation.cc:407] Connection header received
0809 03:35:16.894775 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:16.894778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:16.894828 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:16.894922 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:16.895677 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.896231 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.897205 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:16.897435 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:16.897551 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:16.900044 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:16.900048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:16.900050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:16.900076 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:16.901375 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:16.901820 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:16.901829 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:16.901830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:16.901910 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:16.902198 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:16.902202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:16.902204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:16.902453 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:16.902604 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:16.902850 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 03:35:16.902979 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 03:35:16.903605 7334 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58774
I20260809 03:35:16.904556 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 7449
I20260809 03:35:16.918795 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 7656
I20260809 03:35:16.929128 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 7911
I20260809 03:35:16.944597 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 7287
2026-08-09T03:35:16Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2440 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:17 dist-test-slave-jq4t krb5kdc[8131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:17 dist-test-slave-jq4t krb5kdc[8131](info): set up 2 sockets
Aug 09 03:35:17 dist-test-slave-jq4t krb5kdc[8131](info): commencing operation
krb5kdc: starting...
W20260809 03:35:18.206840 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:35:18 dist-test-slave-jq4t krb5kdc[8131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:35:18.343873 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:40073
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:40073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:18.466152 8147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:18.466468 8147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:18.466550 8147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:18.470650 8147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:18.470731 8147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:18.470767 8147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:18.470803 8147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:18.470852 8147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:18.476336 8147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:40073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:40073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:18.477497 8147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:18.478578 8147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:35:18.488258 8147 server_base.cc:1034] running on GCE node
W20260809 03:35:18.488181 8160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:18.488170 8159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:18.488181 8162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:18.489086 8147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:18.490288 8147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:18.491489 8147 hybrid_clock.cc:648] HybridClock initialized: now 1786246518491459 us; error 43 us; skew 500 ppm
Aug 09 03:35:18 dist-test-slave-jq4t krb5kdc[8131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246518, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:18.494448 8147 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:35:18.495378 8147 webserver.cc:460] Webserver started at http://127.1.224.126:40775/ using document root <none> and password file <none>
I20260809 03:35:18.495990 8147 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:18.496084 8147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:18.496344 8147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:18.498581 8147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "1d57e16f923b4d9fae15acf3c02dffb2"
format_stamp: "Formatted at 2026-08-09 03:35:18 on dist-test-slave-jq4t"
server_key: "c8de209846bb7880db8b5a978fbb7621"
server_key_iv: "150a133e3761fcc9baa58246d985bc5d"
server_key_version: "encryptionkey@0"
I20260809 03:35:18.499203 8147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "1d57e16f923b4d9fae15acf3c02dffb2"
format_stamp: "Formatted at 2026-08-09 03:35:18 on dist-test-slave-jq4t"
server_key: "c8de209846bb7880db8b5a978fbb7621"
server_key_iv: "150a133e3761fcc9baa58246d985bc5d"
server_key_version: "encryptionkey@0"
I20260809 03:35:18.503302 8147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:35:18.506013 8177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:18.507354 8147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:35:18.507519 8147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "1d57e16f923b4d9fae15acf3c02dffb2"
format_stamp: "Formatted at 2026-08-09 03:35:18 on dist-test-slave-jq4t"
server_key: "c8de209846bb7880db8b5a978fbb7621"
server_key_iv: "150a133e3761fcc9baa58246d985bc5d"
server_key_version: "encryptionkey@0"
I20260809 03:35:18.507655 8147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:18.528981 8147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:18.532548 8147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:18.532786 8147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:18.540335 8147 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:40073
I20260809 03:35:18.540416 8259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:40073 every 8 connection(s)
I20260809 03:35:18.541450 8147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:18.541713 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 8147
I20260809 03:35:18.541843 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:18.542207 1921 external_mini_cluster.cc:1428] Setting key e2f40ab26c9152aaf1a170bda5915c0b
I20260809 03:35:18.545245 8262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 03:35:18 dist-test-slave-jq4t krb5kdc[8131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:18.552356 8262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2: Bootstrap starting.
I20260809 03:35:18.555500 8262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:18.556526 8262 log.cc:824] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:18.558300 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:18.543784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46048 (local address 127.1.224.126:40073)
0809 03:35:18.544224 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:18.544246 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:18.544291 (+ 45us) server_negotiation.cc:407] Connection header received
0809 03:35:18.545237 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:18.545264 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:18.545709 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:18.546096 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:18.546826 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:18.548393 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:18.549290 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:18.550007 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:18.550173 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:18.552889 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:18.552895 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:18.552912 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:18.552946 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:18.555043 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:18.555549 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:18.555557 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:18.555564 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:18.555716 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:18.556033 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:18.556036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:18.556038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:18.556648 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:18.556841 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:18.557085 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 03:35:18.557335 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":122,"threads_started":1}
I20260809 03:35:18.559108 8262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2: No bootstrap required, opened a new log
I20260809 03:35:18.563105 8262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d57e16f923b4d9fae15acf3c02dffb2" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40073 } }
I20260809 03:35:18.563326 8262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:18.563369 8262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d57e16f923b4d9fae15acf3c02dffb2, State: Initialized, Role: FOLLOWER
I20260809 03:35:18.563797 8262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d57e16f923b4d9fae15acf3c02dffb2" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40073 } }
I20260809 03:35:18.563952 8262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:18.564042 8262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:18.564145 8262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:18.565306 8262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d57e16f923b4d9fae15acf3c02dffb2" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40073 } }
I20260809 03:35:18.565707 8262 leader_election.cc:302] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d57e16f923b4d9fae15acf3c02dffb2; no voters:
I20260809 03:35:18.566085 8262 leader_election.cc:288] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:18.566217 8270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:18.566463 8270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [term 1 LEADER]: Becoming Leader. State: Replica: 1d57e16f923b4d9fae15acf3c02dffb2, State: Running, Role: LEADER
I20260809 03:35:18.566835 8270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d57e16f923b4d9fae15acf3c02dffb2" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40073 } }
I20260809 03:35:18.567346 8262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:18.568619 8275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d57e16f923b4d9fae15acf3c02dffb2. Latest consensus state: current_term: 1 leader_uuid: "1d57e16f923b4d9fae15acf3c02dffb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d57e16f923b4d9fae15acf3c02dffb2" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40073 } } }
I20260809 03:35:18.568760 8275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:18.569130 8274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d57e16f923b4d9fae15acf3c02dffb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d57e16f923b4d9fae15acf3c02dffb2" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40073 } } }
I20260809 03:35:18.569298 8274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:18.569187 8294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:18.572343 8294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:18.577713 8294 catalog_manager.cc:1347] Generated new cluster ID: 7972672dc6db427194e7995558169664
I20260809 03:35:18.577831 8294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:18.593488 8294 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:18.594471 8294 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:18.600721 8294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d57e16f923b4d9fae15acf3c02dffb2: Generated new TSK 0
I20260809 03:35:18.601558 8294 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:35:18.648332 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40073
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:18.793736 8317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:18.794041 8317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:18.794085 8317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:18.798334 8317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:18.798408 8317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:18.798470 8317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:18.803987 8317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:18.805123 8317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:18.806159 8317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:18.816326 8325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:18.816545 8317 server_base.cc:1034] running on GCE node
W20260809 03:35:18.816318 8328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:18.816318 8324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:18.817159 8317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:18.817777 8317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:18.818941 8317 hybrid_clock.cc:648] HybridClock initialized: now 1786246518818913 us; error 41 us; skew 500 ppm
Aug 09 03:35:18 dist-test-slave-jq4t krb5kdc[8131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246518, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:18.822001 8317 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:35:18.822886 8317 webserver.cc:460] Webserver started at http://127.1.224.65:39927/ using document root <none> and password file <none>
I20260809 03:35:18.823470 8317 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:18.823575 8317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:18.823818 8317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:18.826002 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "ef4279ac6fe14fa4a3c374880d5a2f66"
format_stamp: "Formatted at 2026-08-09 03:35:18 on dist-test-slave-jq4t"
server_key: "2d6c2204596b224d8543f3cb92a9e5c8"
server_key_iv: "8a9ff78bc08de1638384d86945625c12"
server_key_version: "encryptionkey@0"
I20260809 03:35:18.826601 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "ef4279ac6fe14fa4a3c374880d5a2f66"
format_stamp: "Formatted at 2026-08-09 03:35:18 on dist-test-slave-jq4t"
server_key: "2d6c2204596b224d8543f3cb92a9e5c8"
server_key_iv: "8a9ff78bc08de1638384d86945625c12"
server_key_version: "encryptionkey@0"
I20260809 03:35:18.830484 8317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:18.833060 8342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:18.834296 8317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:18.834435 8317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "ef4279ac6fe14fa4a3c374880d5a2f66"
format_stamp: "Formatted at 2026-08-09 03:35:18 on dist-test-slave-jq4t"
server_key: "2d6c2204596b224d8543f3cb92a9e5c8"
server_key_iv: "8a9ff78bc08de1638384d86945625c12"
server_key_version: "encryptionkey@0"
I20260809 03:35:18.834556 8317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:18.857532 8317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:18.860633 8317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:18.860845 8317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:18.861465 8317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:18.862363 8317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:18.862438 8317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:18.862521 8317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:18.862567 8317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:18.872141 8317 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:35665
I20260809 03:35:18.872170 8531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:35665 every 8 connection(s)
I20260809 03:35:18.873152 8317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:18.877838 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 8317
I20260809 03:35:18.877986 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:18.878324 1921 external_mini_cluster.cc:1428] Setting key 0746082e73410867af69d9e1b883cfe2
Aug 09 03:35:18 dist-test-slave-jq4t krb5kdc[8131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246518, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:18.889258 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:18.875156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:40921 (local address 127.1.224.126:40073)
0809 03:35:18.875316 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:18.875321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:18.876297 (+ 976us) server_negotiation.cc:407] Connection header received
0809 03:35:18.877777 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:18.877781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:18.877832 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:18.877907 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:18.879505 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:18.880205 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:18.882043 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:18.882447 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:18.882574 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:18.885389 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:18.885393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:18.885395 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:18.885418 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:18.886965 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:18.887526 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:18.887532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:18.887539 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:18.887618 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:18.887993 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:18.888011 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:18.888012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:18.888199 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:18.888311 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:18.888918 (+ 607us) server_negotiation.cc:299] Negotiation successful
0809 03:35:18.889069 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:35:18.890161 8536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:18.875633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:40073 (local address 127.1.224.65:40921)
0809 03:35:18.876124 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:18.876171 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:18.877530 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:18.878126 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:18.878136 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:18.878664 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:18.879343 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:18.879361 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:18.880352 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:18.880356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:18.881876 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:18.881892 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:18.882600 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:18.882603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:18.882747 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:18.883399 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:18.883416 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:18.885210 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:18.887104 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:18.887111 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:18.887127 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:18.887410 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:18.887717 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:18.887720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:18.887722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:18.887884 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:18.888313 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:18.888321 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:18.888663 (+ 342us) client_negotiation.cc:769] Sending connection context
0809 03:35:18.888857 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 03:35:18.889218 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":122,"threads_started":1}
I20260809 03:35:18.891564 8532 heartbeater.cc:347] Connected to a master server at 127.1.224.126:40073
I20260809 03:35:18.891834 8532 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:18.892381 8532 heartbeater.cc:511] Master 127.1.224.126:40073 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:35:18.894043 8205 ts_manager.cc:194] Registered new tserver with Master: ef4279ac6fe14fa4a3c374880d5a2f66 (127.1.224.65:35665)
I20260809 03:35:18.895710 8205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:40921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:35:18.909850 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40073
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:19.027494 8540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:19.027750 8540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:19.027794 8540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:19.031847 8540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:19.031913 8540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:19.031971 8540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:19.037477 8540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:19.038759 8540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:19.039861 8540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:19.050215 8550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:19.050349 8540 server_base.cc:1034] running on GCE node
W20260809 03:35:19.050275 8553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:19.050262 8555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:19.050908 8540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:19.051630 8540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:19.052796 8540 hybrid_clock.cc:648] HybridClock initialized: now 1786246519052759 us; error 50 us; skew 500 ppm
Aug 09 03:35:19 dist-test-slave-jq4t krb5kdc[8131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246519, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:19.055774 8540 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:35:19.056895 8540 webserver.cc:460] Webserver started at http://127.1.224.66:38445/ using document root <none> and password file <none>
I20260809 03:35:19.057598 8540 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:19.057722 8540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:19.057991 8540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:19.060146 8540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "8dd5ea1d9ba34a0cbecf91c5aaa1cb3b"
format_stamp: "Formatted at 2026-08-09 03:35:19 on dist-test-slave-jq4t"
server_key: "2f5b998fb4c9b92d26a5d067b90f85e0"
server_key_iv: "bb5af721b7808d8c8f2d137808307d02"
server_key_version: "encryptionkey@0"
I20260809 03:35:19.060755 8540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "8dd5ea1d9ba34a0cbecf91c5aaa1cb3b"
format_stamp: "Formatted at 2026-08-09 03:35:19 on dist-test-slave-jq4t"
server_key: "2f5b998fb4c9b92d26a5d067b90f85e0"
server_key_iv: "bb5af721b7808d8c8f2d137808307d02"
server_key_version: "encryptionkey@0"
I20260809 03:35:19.064669 8540 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:35:19.067087 8567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:19.068452 8540 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:19.068655 8540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "8dd5ea1d9ba34a0cbecf91c5aaa1cb3b"
format_stamp: "Formatted at 2026-08-09 03:35:19 on dist-test-slave-jq4t"
server_key: "2f5b998fb4c9b92d26a5d067b90f85e0"
server_key_iv: "bb5af721b7808d8c8f2d137808307d02"
server_key_version: "encryptionkey@0"
I20260809 03:35:19.068753 8540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:19.089048 8540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:19.092231 8540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:19.092404 8540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:19.093094 8540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:19.093990 8540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:19.094038 8540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:19.094080 8540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:19.094102 8540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:19.103674 8540 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:44011
I20260809 03:35:19.103694 8753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:44011 every 8 connection(s)
I20260809 03:35:19.104765 8540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:19.107707 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 8540
I20260809 03:35:19.107829 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:19.108189 1921 external_mini_cluster.cc:1428] Setting key 0571b3a59ee393070c8ffa4d9325afca
Aug 09 03:35:19 dist-test-slave-jq4t krb5kdc[8131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246519, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:35:19.122094 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:19.106767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:43685 (local address 127.1.224.126:40073)
0809 03:35:19.106965 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:19.106970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:19.108020 (+ 1050us) server_negotiation.cc:407] Connection header received
0809 03:35:19.109878 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:19.109881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:19.109946 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:19.110017 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:19.111882 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:19.112633 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:19.114377 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:19.114788 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:19.114920 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:19.118002 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:19.118005 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:19.118007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:19.118034 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:19.119661 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:19.120357 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:19.120361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:19.120363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:19.120466 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:19.120891 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:19.120893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:19.120894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:19.121109 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:19.121189 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:19.121830 (+ 641us) server_negotiation.cc:299] Negotiation successful
0809 03:35:19.121948 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
I20260809 03:35:19.123204 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:19.107250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:40073 (local address 127.1.224.66:43685)
0809 03:35:19.107824 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:19.107891 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:19.109670 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:19.110243 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:19.110252 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:19.110857 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:19.111679 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:19.111703 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:19.112769 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:19.112776 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:19.114233 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:19.114244 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:19.114950 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:19.114953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:19.115124 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:19.115886 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:19.115903 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:19.117766 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:19.119793 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:19.119801 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:19.119820 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:19.120227 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:19.120607 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:19.120609 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:19.120611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:19.120797 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:19.121292 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:19.121299 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:19.121601 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 03:35:19.121775 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 03:35:19.122071 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":130,"threads_started":1}
I20260809 03:35:19.124598 8755 heartbeater.cc:347] Connected to a master server at 127.1.224.126:40073
I20260809 03:35:19.124887 8755 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:19.125509 8755 heartbeater.cc:511] Master 127.1.224.126:40073 requested a full tablet report, sending...
I20260809 03:35:19.126821 8205 ts_manager.cc:194] Registered new tserver with Master: 8dd5ea1d9ba34a0cbecf91c5aaa1cb3b (127.1.224.66:44011)
I20260809 03:35:19.127451 8205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:43685
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:35:19.139850 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40073
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:19.259991 8765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:19.260290 8765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:19.260383 8765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:19.264564 8765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:19.264642 8765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:19.264752 8765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:19.270668 8765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:19.272032 8765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:19.273379 8765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:19.283713 8771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:19.283716 8774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:19.283854 8765 server_base.cc:1034] running on GCE node
W20260809 03:35:19.283716 8772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:19.284456 8765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:19.285220 8765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:19.286434 8765 hybrid_clock.cc:648] HybridClock initialized: now 1786246519286411 us; error 45 us; skew 500 ppm
Aug 09 03:35:19 dist-test-slave-jq4t krb5kdc[8131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246519, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:19.289523 8765 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:35:19.290480 8765 webserver.cc:460] Webserver started at http://127.1.224.67:44725/ using document root <none> and password file <none>
I20260809 03:35:19.291065 8765 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:19.291131 8765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:19.291306 8765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:19.293515 8765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "1274b649cd394c30b64327185d9b992e"
format_stamp: "Formatted at 2026-08-09 03:35:19 on dist-test-slave-jq4t"
server_key: "d6048c67b4bb0ff9feb1fcf297b1f01a"
server_key_iv: "08051be037efe5837affaf1d0be4cf70"
server_key_version: "encryptionkey@0"
I20260809 03:35:19.294063 8765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "1274b649cd394c30b64327185d9b992e"
format_stamp: "Formatted at 2026-08-09 03:35:19 on dist-test-slave-jq4t"
server_key: "d6048c67b4bb0ff9feb1fcf297b1f01a"
server_key_iv: "08051be037efe5837affaf1d0be4cf70"
server_key_version: "encryptionkey@0"
I20260809 03:35:19.297987 8765 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:35:19.300429 8784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:19.301793 8765 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:35:19.301901 8765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "1274b649cd394c30b64327185d9b992e"
format_stamp: "Formatted at 2026-08-09 03:35:19 on dist-test-slave-jq4t"
server_key: "d6048c67b4bb0ff9feb1fcf297b1f01a"
server_key_iv: "08051be037efe5837affaf1d0be4cf70"
server_key_version: "encryptionkey@0"
I20260809 03:35:19.302055 8765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:19.322640 8765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:19.326122 8765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:19.326347 8765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:19.327000 8765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:19.327868 8765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:19.327915 8765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:19.327984 8765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:19.328018 8765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:19.337396 8765 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:37597
I20260809 03:35:19.337416 8967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:37597 every 8 connection(s)
I20260809 03:35:19.338414 8765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:19.341099 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 8765
I20260809 03:35:19.341251 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:19.341539 1921 external_mini_cluster.cc:1428] Setting key fc2ea64d9e9125d3d49bd6d8bd9bda30
Aug 09 03:35:19 dist-test-slave-jq4t krb5kdc[8131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246519, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:19.353119 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:19.340421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:52195 (local address 127.1.224.126:40073)
0809 03:35:19.340631 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:19.340636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:19.341526 (+ 890us) server_negotiation.cc:407] Connection header received
0809 03:35:19.343251 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:19.343255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:19.343314 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:19.343392 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:19.344700 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:19.345256 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:19.346442 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:19.346676 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:19.346791 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:19.349350 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:19.349354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:19.349356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:19.349379 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:19.350869 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:19.351412 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:19.351416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:19.351417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:19.351496 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:19.351852 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:19.351856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:19.351858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:19.352059 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:19.352178 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:19.352784 (+ 606us) server_negotiation.cc:299] Negotiation successful
0809 03:35:19.352926 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260809 03:35:19.353945 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:19.340871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:40073 (local address 127.1.224.67:52195)
0809 03:35:19.341353 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:19.341410 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:19.343044 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:19.343566 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:19.343592 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:19.344000 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:19.344563 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:19.344579 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:19.345414 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:19.345417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:19.346312 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:19.346322 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:19.346834 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:19.346839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:19.347016 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:19.347648 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:19.347665 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:19.349176 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:19.350997 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:19.351005 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:19.351021 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:19.351291 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:19.351591 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:19.351594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:19.351596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:19.351745 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:19.352176 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:19.352184 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:19.352455 (+ 271us) client_negotiation.cc:769] Sending connection context
0809 03:35:19.352724 (+ 269us) client_negotiation.cc:240] Negotiation successful
0809 03:35:19.353079 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":133,"threads_started":1}
I20260809 03:35:19.355183 8968 heartbeater.cc:347] Connected to a master server at 127.1.224.126:40073
I20260809 03:35:19.355441 8968 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:19.356060 8968 heartbeater.cc:511] Master 127.1.224.126:40073 requested a full tablet report, sending...
I20260809 03:35:19.357090 8197 ts_manager.cc:194] Registered new tserver with Master: 1274b649cd394c30b64327185d9b992e (127.1.224.67:37597)
I20260809 03:35:19.357647 8197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:52195
I20260809 03:35:19.367641 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:35:19 dist-test-slave-jq4t krb5kdc[8131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:35:19 dist-test-slave-jq4t krb5kdc[8131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.65@KRBTEST.COM
I20260809 03:35:19.404829 8980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:19.389141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.1.224.65:35665)
0809 03:35:19.389544 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:19.389560 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:19.389589 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:35:19.389648 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:19.389654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:19.389870 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:19.390019 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:19.390969 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:19.391771 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:19.392771 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:19.393191 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:19.393282 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:19.400868 (+ 7586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:19.400888 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:19.400908 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:19.400948 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:19.402820 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:19.403343 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:19.403349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:19.403350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:19.403439 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:19.403696 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:19.403698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:19.403699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:19.403999 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:19.404174 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:19.404359 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 03:35:19.404602 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":161,"threads_started":1}
Aug 09 03:35:19 dist-test-slave-jq4t krb5kdc[8131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:19.417349 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:19.408153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46050 (local address 127.1.224.126:40073)
0809 03:35:19.408308 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:19.408311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:19.408331 (+ 20us) server_negotiation.cc:407] Connection header received
0809 03:35:19.408565 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:19.408569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:19.408626 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:19.408707 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:19.409360 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:19.409925 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:19.410795 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:19.411065 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:19.411169 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:19.413884 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:19.413888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:19.413890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:19.413920 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:19.415443 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:19.415892 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:19.415897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:19.415899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:19.415997 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:19.416292 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:19.416295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:19.416297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:19.416633 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:19.416781 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:19.417042 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 03:35:19.417173 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 03:35:19.417814 8197 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46050
I20260809 03:35:19.418608 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 8317
I20260809 03:35:19.430686 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 8540
I20260809 03:35:19.444204 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 8765
I20260809 03:35:19.466833 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 8147
2026-08-09T03:35:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2520 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:19 dist-test-slave-jq4t krb5kdc[8988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:19 dist-test-slave-jq4t krb5kdc[8988](info): set up 2 sockets
Aug 09 03:35:19 dist-test-slave-jq4t krb5kdc[8988](info): commencing operation
krb5kdc: starting...
W20260809 03:35:20.730278 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:35:20 dist-test-slave-jq4t krb5kdc[8988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:35:20.868248 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:43699
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:43699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:20.984315 9011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:20.984704 9011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:20.984802 9011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:20.989059 9011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:20.989183 9011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:20.989220 9011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:20.989262 9011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:20.989298 9011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:20.994910 9011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:43699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:43699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:20.996258 9011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:20.997520 9011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:35:21.006583 9011 server_base.cc:1034] running on GCE node
W20260809 03:35:21.006569 9021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:21.006538 9018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:21.006538 9019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:21.007403 9011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:21.008646 9011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:21.009842 9011 hybrid_clock.cc:648] HybridClock initialized: now 1786246521009829 us; error 31 us; skew 500 ppm
Aug 09 03:35:21 dist-test-slave-jq4t krb5kdc[8988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:21.012796 9011 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:35:21.013626 9011 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:21.013725 9011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:21.013993 9011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:21.016255 9011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "caf32035b2c94de884b38385162827ee"
format_stamp: "Formatted at 2026-08-09 03:35:21 on dist-test-slave-jq4t"
server_key: "f916af86bd2b1d0a2f0ddfb06f105cf0"
server_key_iv: "82d394270fec7ca7cfef64f9b4988ffd"
server_key_version: "encryptionkey@0"
I20260809 03:35:21.016916 9011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "caf32035b2c94de884b38385162827ee"
format_stamp: "Formatted at 2026-08-09 03:35:21 on dist-test-slave-jq4t"
server_key: "f916af86bd2b1d0a2f0ddfb06f105cf0"
server_key_iv: "82d394270fec7ca7cfef64f9b4988ffd"
server_key_version: "encryptionkey@0"
I20260809 03:35:21.020809 9011 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:35:21.023396 9034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:21.024871 9011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:21.025035 9011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "caf32035b2c94de884b38385162827ee"
format_stamp: "Formatted at 2026-08-09 03:35:21 on dist-test-slave-jq4t"
server_key: "f916af86bd2b1d0a2f0ddfb06f105cf0"
server_key_iv: "82d394270fec7ca7cfef64f9b4988ffd"
server_key_version: "encryptionkey@0"
I20260809 03:35:21.025156 9011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:21.045254 9011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:21.049062 9011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:21.049311 9011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:21.057771 9011 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:43699
I20260809 03:35:21.058143 9116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:43699 every 8 connection(s)
I20260809 03:35:21.058763 9011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:21.062031 9119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:21.066187 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 9011
I20260809 03:35:21.066308 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:21.066594 1921 external_mini_cluster.cc:1428] Setting key d33c85ac970137200527f59a453a76da
I20260809 03:35:21.069195 9119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee: Bootstrap starting.
I20260809 03:35:21.072229 9119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:21.073201 9119 log.cc:824] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:21.075500 9119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee: No bootstrap required, opened a new log
Aug 09 03:35:21 dist-test-slave-jq4t krb5kdc[8988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:21.079787 9119 raft_consensus.cc:348] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf32035b2c94de884b38385162827ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43699 } }
I20260809 03:35:21.080113 9119 raft_consensus.cc:374] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:21.080183 9119 raft_consensus.cc:729] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf32035b2c94de884b38385162827ee, State: Initialized, Role: FOLLOWER
I20260809 03:35:21.080746 9119 consensus_queue.cc:260] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf32035b2c94de884b38385162827ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43699 } }
I20260809 03:35:21.080919 9119 raft_consensus.cc:388] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:21.080992 9119 raft_consensus.cc:482] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:21.081102 9119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:21.082455 9119 raft_consensus.cc:504] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf32035b2c94de884b38385162827ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43699 } }
I20260809 03:35:21.082870 9119 leader_election.cc:302] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caf32035b2c94de884b38385162827ee; no voters:
I20260809 03:35:21.083268 9119 leader_election.cc:288] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:21.083405 9124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:21.083667 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.068097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.1.224.126:43699)
0809 03:35:21.068583 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:21.068606 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:21.068656 (+ 50us) server_negotiation.cc:407] Connection header received
0809 03:35:21.069559 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:21.069587 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:21.070045 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:21.070419 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:21.071183 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.072756 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.073741 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.074355 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.074536 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.077200 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:21.077207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:21.077226 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:21.077262 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:21.079978 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:21.080625 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:21.080632 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:21.080637 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:21.080766 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:21.081108 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:21.081115 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:21.081117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:21.081695 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:21.081919 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:21.082211 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 03:35:21.082568 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":120,"threads_started":1}
I20260809 03:35:21.083671 9124 raft_consensus.cc:686] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [term 1 LEADER]: Becoming Leader. State: Replica: caf32035b2c94de884b38385162827ee, State: Running, Role: LEADER
I20260809 03:35:21.084429 9124 consensus_queue.cc:237] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf32035b2c94de884b38385162827ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43699 } }
I20260809 03:35:21.084776 9119 sys_catalog.cc:564] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:21.086426 9127 sys_catalog.cc:455] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caf32035b2c94de884b38385162827ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf32035b2c94de884b38385162827ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43699 } } }
I20260809 03:35:21.086602 9127 sys_catalog.cc:458] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:21.086854 9126 sys_catalog.cc:455] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader caf32035b2c94de884b38385162827ee. Latest consensus state: current_term: 1 leader_uuid: "caf32035b2c94de884b38385162827ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf32035b2c94de884b38385162827ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43699 } } }
I20260809 03:35:21.086963 9126 sys_catalog.cc:458] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:21.087200 9147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:21.090495 9147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:21.096191 9147 catalog_manager.cc:1347] Generated new cluster ID: a3d7462fc4ef4a58a4bf55d5a484426f
I20260809 03:35:21.096273 9147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:21.117358 9147 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:21.118503 9147 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:21.124022 9147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee: Generated new TSK 0
I20260809 03:35:21.124867 9147 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:35:21.164057 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43699
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:21.283319 9161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:21.283667 9161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:21.283751 9161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:21.288447 9161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:21.288584 9161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:21.288690 9161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:21.294574 9161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:21.295849 9161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:21.297057 9161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:21.306914 9170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:21.306915 9172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:21.306922 9169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:21.307343 9161 server_base.cc:1034] running on GCE node
I20260809 03:35:21.307806 9161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:21.308455 9161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:21.309756 9161 hybrid_clock.cc:648] HybridClock initialized: now 1786246521309742 us; error 31 us; skew 500 ppm
Aug 09 03:35:21 dist-test-slave-jq4t krb5kdc[8988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:21.313187 9161 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:35:21.314097 9161 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:21.314188 9161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:21.314440 9161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:21.316867 9161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "6464983695c247ac838ef6adfd614cf9"
format_stamp: "Formatted at 2026-08-09 03:35:21 on dist-test-slave-jq4t"
server_key: "fcc187795a9dcb9999d2703777128c84"
server_key_iv: "81a08c4726de744f0ec0bc69df6faa3f"
server_key_version: "encryptionkey@0"
I20260809 03:35:21.317524 9161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "6464983695c247ac838ef6adfd614cf9"
format_stamp: "Formatted at 2026-08-09 03:35:21 on dist-test-slave-jq4t"
server_key: "fcc187795a9dcb9999d2703777128c84"
server_key_iv: "81a08c4726de744f0ec0bc69df6faa3f"
server_key_version: "encryptionkey@0"
I20260809 03:35:21.321789 9161 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:35:21.324651 9181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:21.326164 9161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:21.326331 9161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "6464983695c247ac838ef6adfd614cf9"
format_stamp: "Formatted at 2026-08-09 03:35:21 on dist-test-slave-jq4t"
server_key: "fcc187795a9dcb9999d2703777128c84"
server_key_iv: "81a08c4726de744f0ec0bc69df6faa3f"
server_key_version: "encryptionkey@0"
I20260809 03:35:21.326489 9161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:21.353652 9161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:21.357039 9161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:21.357285 9161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:21.357952 9161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:21.358903 9161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:21.358982 9161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:21.359064 9161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:21.359110 9161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:21.368868 9161 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:37019
I20260809 03:35:21.368937 9358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:37019 every 8 connection(s)
I20260809 03:35:21.369572 9161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:21.372867 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 9161
I20260809 03:35:21.373001 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:21.373308 1921 external_mini_cluster.cc:1428] Setting key d6ebad5370b7e1b3b3f85a1d5d38a6ae
Aug 09 03:35:21 dist-test-slave-jq4t krb5kdc[8988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:35:21.386782 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.371507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:41623 (local address 127.1.224.126:43699)
0809 03:35:21.371665 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:21.371669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:21.372606 (+ 937us) server_negotiation.cc:407] Connection header received
0809 03:35:21.374115 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:21.374118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:21.374165 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:21.374226 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:21.375550 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.376107 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.377371 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.377610 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.377729 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.381057 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:21.381064 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:21.381067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:21.381101 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:21.383266 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:21.384164 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:21.384170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:21.384173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:21.384294 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:21.384868 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:21.384873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:21.384875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:21.385182 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:21.385332 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:21.386361 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0809 03:35:21.386585 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:35:21.387995 9361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.371893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:43699 (local address 127.1.224.65:41623)
0809 03:35:21.372388 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:21.372445 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:21.373900 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:21.374403 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:21.374423 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:21.374865 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:21.375387 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:21.375402 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.376258 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.376261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:21.377238 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:21.377248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.377776 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.377781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:21.377959 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.378617 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:21.378634 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:21.380719 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:21.383442 (+ 2723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:21.383450 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:21.383504 (+ 54us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:21.383992 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:21.384442 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:21.384448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:21.384452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:21.384746 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:21.385399 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:21.385408 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:21.385923 (+ 515us) client_negotiation.cc:769] Sending connection context
0809 03:35:21.386217 (+ 294us) client_negotiation.cc:240] Negotiation successful
0809 03:35:21.386684 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:35:21.390014 9359 heartbeater.cc:347] Connected to a master server at 127.1.224.126:43699
I20260809 03:35:21.390312 9359 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:21.390905 9359 heartbeater.cc:511] Master 127.1.224.126:43699 requested a full tablet report, sending...
I20260809 03:35:21.392935 9065 ts_manager.cc:194] Registered new tserver with Master: 6464983695c247ac838ef6adfd614cf9 (127.1.224.65:37019)
I20260809 03:35:21.394814 9065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:41623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:35:21.407325 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43699
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:21.524916 9365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:21.525209 9365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:21.525259 9365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:21.529841 9365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:21.529923 9365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:21.529991 9365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:21.535905 9365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:21.537263 9365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:21.538437 9365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:21.548821 9375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:21.548842 9373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:21.548826 9377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:21.549234 9365 server_base.cc:1034] running on GCE node
I20260809 03:35:21.549721 9365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:21.550402 9365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:21.551584 9365 hybrid_clock.cc:648] HybridClock initialized: now 1786246521551567 us; error 29 us; skew 500 ppm
Aug 09 03:35:21 dist-test-slave-jq4t krb5kdc[8988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:21.554718 9365 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:35:21.555559 9365 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:21.555625 9365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:21.555867 9365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:21.558166 9365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "cda3bc309d6745e08b9a07ac83c7f34a"
format_stamp: "Formatted at 2026-08-09 03:35:21 on dist-test-slave-jq4t"
server_key: "7054231a82195ff13c2330dc27a15afc"
server_key_iv: "5f945bb89756f62e10c3f2e926fa78a5"
server_key_version: "encryptionkey@0"
I20260809 03:35:21.558785 9365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "cda3bc309d6745e08b9a07ac83c7f34a"
format_stamp: "Formatted at 2026-08-09 03:35:21 on dist-test-slave-jq4t"
server_key: "7054231a82195ff13c2330dc27a15afc"
server_key_iv: "5f945bb89756f62e10c3f2e926fa78a5"
server_key_version: "encryptionkey@0"
I20260809 03:35:21.562803 9365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:21.565341 9385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:21.566511 9365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:21.566658 9365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "cda3bc309d6745e08b9a07ac83c7f34a"
format_stamp: "Formatted at 2026-08-09 03:35:21 on dist-test-slave-jq4t"
server_key: "7054231a82195ff13c2330dc27a15afc"
server_key_iv: "5f945bb89756f62e10c3f2e926fa78a5"
server_key_version: "encryptionkey@0"
I20260809 03:35:21.566768 9365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:21.582865 9365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:21.586356 9365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:21.586611 9365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:21.587291 9365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:21.588215 9365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:21.588290 9365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:21.588367 9365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:21.588419 9365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:21.597883 9365 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:34743
I20260809 03:35:21.597901 9565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:34743 every 8 connection(s)
I20260809 03:35:21.598573 9365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:21.605298 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 9365
I20260809 03:35:21.605399 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:21.605693 1921 external_mini_cluster.cc:1428] Setting key 5a7e0930a83375db16091af60d8b70d6
Aug 09 03:35:21 dist-test-slave-jq4t krb5kdc[8988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:21.613866 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.600429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:41353 (local address 127.1.224.126:43699)
0809 03:35:21.600650 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:21.600655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:21.601607 (+ 952us) server_negotiation.cc:407] Connection header received
0809 03:35:21.602913 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:21.602917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:21.602969 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:21.603048 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:21.604248 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.604865 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.606311 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.606602 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.606688 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.609527 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:21.609534 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:21.609538 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:21.609572 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:21.611299 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:21.611969 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:21.611972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:21.611973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:21.612058 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:21.612554 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:21.612558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:21.612559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:21.612829 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:21.612914 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:21.613624 (+ 710us) server_negotiation.cc:299] Negotiation successful
0809 03:35:21.613737 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":98}
I20260809 03:35:21.615236 9568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.600887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:43699 (local address 127.1.224.66:41353)
0809 03:35:21.601412 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:21.601461 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:21.602710 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:21.603221 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:21.603227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:21.603594 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:21.604104 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:21.604118 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.605012 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.605015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:21.606135 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:21.606150 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.606771 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.606775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:21.607007 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.607744 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:21.607761 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:21.609344 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:21.611435 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:21.611445 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:21.611465 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:21.611868 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:21.612154 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:21.612160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:21.612165 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:21.612389 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:21.612954 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:21.612964 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:21.613326 (+ 362us) client_negotiation.cc:769] Sending connection context
0809 03:35:21.613576 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 03:35:21.613990 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":134,"threads_started":1}
I20260809 03:35:21.616556 9566 heartbeater.cc:347] Connected to a master server at 127.1.224.126:43699
I20260809 03:35:21.616820 9566 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:21.617317 9566 heartbeater.cc:511] Master 127.1.224.126:43699 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:35:21.618538 9065 ts_manager.cc:194] Registered new tserver with Master: cda3bc309d6745e08b9a07ac83c7f34a (127.1.224.66:34743)
I20260809 03:35:21.619159 9065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:41353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:35:21.635667 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43699
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:21.752178 9571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:21.752445 9571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:21.752511 9571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:21.756565 9571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:21.756623 9571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:21.756732 9571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:21.761988 9571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:21.763091 9571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:21.764202 9571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:21.774983 9579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:21.774995 9581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:21.775000 9577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:21.775259 9571 server_base.cc:1034] running on GCE node
I20260809 03:35:21.775924 9571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:21.776708 9571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:21.777901 9571 hybrid_clock.cc:648] HybridClock initialized: now 1786246521777884 us; error 29 us; skew 500 ppm
Aug 09 03:35:21 dist-test-slave-jq4t krb5kdc[8988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:21.780994 9571 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:35:21.781728 9571 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:21.781791 9571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:21.782043 9571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:21.784339 9571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "c54e5e423496472dada8e684fdec1cb5"
format_stamp: "Formatted at 2026-08-09 03:35:21 on dist-test-slave-jq4t"
server_key: "5abebdc22f0c6ea91183b5a37341d9bb"
server_key_iv: "9eed572a1f558d770ce72288fbde4f37"
server_key_version: "encryptionkey@0"
I20260809 03:35:21.785219 9571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "c54e5e423496472dada8e684fdec1cb5"
format_stamp: "Formatted at 2026-08-09 03:35:21 on dist-test-slave-jq4t"
server_key: "5abebdc22f0c6ea91183b5a37341d9bb"
server_key_iv: "9eed572a1f558d770ce72288fbde4f37"
server_key_version: "encryptionkey@0"
I20260809 03:35:21.789592 9571 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 03:35:21.791975 9591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:21.793279 9571 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:35:21.793385 9571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "c54e5e423496472dada8e684fdec1cb5"
format_stamp: "Formatted at 2026-08-09 03:35:21 on dist-test-slave-jq4t"
server_key: "5abebdc22f0c6ea91183b5a37341d9bb"
server_key_iv: "9eed572a1f558d770ce72288fbde4f37"
server_key_version: "encryptionkey@0"
I20260809 03:35:21.793470 9571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:21.818292 9571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:21.821776 9571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:21.821951 9571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:21.822515 9571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:21.823388 9571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:21.823438 9571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:21.823479 9571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:21.823501 9571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:21.833449 9571 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:37975
I20260809 03:35:21.833464 9781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:37975 every 8 connection(s)
I20260809 03:35:21.834178 9571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:21.843130 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 9571
I20260809 03:35:21.843290 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:21.843668 1921 external_mini_cluster.cc:1428] Setting key 709497e8052644833ba99f89596bf391
Aug 09 03:35:21 dist-test-slave-jq4t krb5kdc[8988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246521, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:21.849084 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.836297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:54395 (local address 127.1.224.126:43699)
0809 03:35:21.836460 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:21.836464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:21.837542 (+ 1078us) server_negotiation.cc:407] Connection header received
0809 03:35:21.838980 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:21.838984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:21.839060 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:21.839178 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:21.840512 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.841097 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.842342 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.842601 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.842725 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.845602 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:21.845606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:21.845609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:21.845640 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:21.847097 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:21.847615 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:21.847617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:21.847619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:21.847694 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:21.848001 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:21.848003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:21.848005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:21.848191 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:21.848269 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:21.848850 (+ 581us) server_negotiation.cc:299] Negotiation successful
0809 03:35:21.848962 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:35:21.849961 9784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.836855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:43699 (local address 127.1.224.67:54395)
0809 03:35:21.837376 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:21.837421 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:21.838741 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:21.839385 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:21.839392 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:21.839793 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:21.840314 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:21.840327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.841243 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.841249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:21.842194 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:21.842204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.842779 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.842783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:21.842970 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.843698 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:21.843740 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:21.845417 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:21.847209 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:21.847214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:21.847228 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:21.847522 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:21.847775 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:21.847777 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:21.847779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:21.847921 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:21.848286 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:21.848291 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:21.848626 (+ 335us) client_negotiation.cc:769] Sending connection context
0809 03:35:21.848798 (+ 172us) client_negotiation.cc:240] Negotiation successful
0809 03:35:21.849077 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":113,"threads_started":1}
I20260809 03:35:21.851126 9782 heartbeater.cc:347] Connected to a master server at 127.1.224.126:43699
I20260809 03:35:21.851388 9782 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:21.851869 9782 heartbeater.cc:511] Master 127.1.224.126:43699 requested a full tablet report, sending...
I20260809 03:35:21.852972 9065 ts_manager.cc:194] Registered new tserver with Master: c54e5e423496472dada8e684fdec1cb5 (127.1.224.67:37975)
I20260809 03:35:21.853513 9065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:54395
I20260809 03:35:21.857978 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:35:21.866824 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.859920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.1.224.126:43699)
0809 03:35:21.860061 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:21.860064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:21.860256 (+ 192us) server_negotiation.cc:407] Connection header received
0809 03:35:21.860342 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:21.860344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:21.860380 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:21.860447 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:21.861234 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.861731 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.862517 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.862711 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.862784 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.864007 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:21.864010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:21.864012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:21.864033 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:21.865490 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:21.865851 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:21.865853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:21.865854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:21.865925 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:21.866157 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:21.866160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:21.866161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:21.866360 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:21.866461 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:21.866626 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 03:35:21.866712 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 03:35:21.869731 9065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:21.873086 9065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:21.888355 9798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.881798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:34743 (local address 127.0.0.1:51402)
0809 03:35:21.882182 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:21.882235 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:21.882377 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:21.883615 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:21.883621 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:21.883645 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:21.883804 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:21.883812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.885394 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.885397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:21.886757 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:21.886768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.887845 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.887849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:21.887969 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.888008 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 03:35:21.888126 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 03:35:21.888227 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":93,"thread_start_us":84,"threads_started":1}
I20260809 03:35:21.888697 9800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.882202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:37975 (local address 127.0.0.1:42492)
0809 03:35:21.882728 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:21.882742 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:21.882828 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:21.883801 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:21.883803 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:21.883819 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:21.883924 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:21.883930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.885238 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.885244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:21.887003 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:21.887013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.888246 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.888249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:21.888340 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.888365 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 03:35:21.888440 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 03:35:21.888546 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":157,"thread_start_us":89,"threads_started":1}
I20260809 03:35:21.888685 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.881943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51402 (local address 127.1.224.66:34743)
0809 03:35:21.883073 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:21.883088 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:21.883117 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:35:21.883206 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:21.883212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:21.883437 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:21.883607 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:21.883938 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.885247 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.886884 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.887696 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.887814 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.887989 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:21.888184 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 03:35:21.888290 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1003,"thread_start_us":123,"threads_started":1}
I20260809 03:35:21.889180 9803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.882663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42492 (local address 127.1.224.67:37975)
0809 03:35:21.883079 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:21.883088 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:21.883132 (+ 44us) server_negotiation.cc:407] Connection header received
0809 03:35:21.883215 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:21.883222 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:21.883439 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:21.883624 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:21.884046 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.885103 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.887145 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.887917 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.888038 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.888181 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:21.888881 (+ 700us) server_negotiation.cc:299] Negotiation successful
0809 03:35:21.889009 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":103,"threads_started":1}
I20260809 03:35:21.889752 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.882571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:37019 (local address 127.0.0.1:39910)
0809 03:35:21.883238 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:21.883251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:21.883338 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:21.884012 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:21.884015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:21.884034 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:21.884119 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:21.884124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.885608 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.885614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:21.887001 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:21.887013 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.888790 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.888796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:21.888916 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.889467 (+ 551us) client_negotiation.cc:769] Sending connection context
0809 03:35:21.889545 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 03:35:21.889629 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":74,"thread_start_us":99,"threads_started":1}
I20260809 03:35:21.889822 9804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:21.882745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.1.224.65:37019)
0809 03:35:21.883585 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:21.883591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:21.883612 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:35:21.883687 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:21.883693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:21.883853 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:21.884036 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:21.884203 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.885449 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.887167 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:21.887708 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:21.887848 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:21.888002 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:21.889578 (+ 1576us) server_negotiation.cc:299] Negotiation successful
0809 03:35:21.889681 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":85,"threads_started":1}
I20260809 03:35:21.892436 9683 tablet_service.cc:1467] Processing CreateTablet for tablet b1b397a5457e4aa082e7ed9a7f3043f5 (DEFAULT_TABLE table=test-table [id=dcec99db94c84d619993e6c78ed28f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:21.892710 9249 tablet_service.cc:1467] Processing CreateTablet for tablet b1b397a5457e4aa082e7ed9a7f3043f5 (DEFAULT_TABLE table=test-table [id=dcec99db94c84d619993e6c78ed28f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:21.893769 9249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1b397a5457e4aa082e7ed9a7f3043f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:21.893774 9683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1b397a5457e4aa082e7ed9a7f3043f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:21.894562 9460 tablet_service.cc:1467] Processing CreateTablet for tablet b1b397a5457e4aa082e7ed9a7f3043f5 (DEFAULT_TABLE table=test-table [id=dcec99db94c84d619993e6c78ed28f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:21.895496 9460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1b397a5457e4aa082e7ed9a7f3043f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:21.901654 9806 tablet_bootstrap.cc:492] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9: Bootstrap starting.
I20260809 03:35:21.902570 9808 tablet_bootstrap.cc:492] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5: Bootstrap starting.
I20260809 03:35:21.903648 9807 tablet_bootstrap.cc:492] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a: Bootstrap starting.
I20260809 03:35:21.904641 9806 tablet_bootstrap.cc:654] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:21.904857 9808 tablet_bootstrap.cc:654] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:21.905615 9806 log.cc:824] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:21.906325 9808 log.cc:824] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:21.906656 9807 tablet_bootstrap.cc:654] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:21.907451 9807 log.cc:824] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:21.908074 9806 tablet_bootstrap.cc:492] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9: No bootstrap required, opened a new log
I20260809 03:35:21.908318 9806 ts_tablet_manager.cc:1397] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260809 03:35:21.908360 9808 tablet_bootstrap.cc:492] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5: No bootstrap required, opened a new log
I20260809 03:35:21.908603 9808 ts_tablet_manager.cc:1397] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:35:21.909520 9807 tablet_bootstrap.cc:492] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a: No bootstrap required, opened a new log
I20260809 03:35:21.909708 9807 ts_tablet_manager.cc:1397] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:35:21.912124 9808 raft_consensus.cc:348] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34743 } } peers { permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 } } peers { permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 } }
I20260809 03:35:21.912398 9808 raft_consensus.cc:374] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:21.912487 9808 raft_consensus.cc:729] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c54e5e423496472dada8e684fdec1cb5, State: Initialized, Role: FOLLOWER
I20260809 03:35:21.912999 9808 consensus_queue.cc:260] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34743 } } peers { permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 } } peers { permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 } }
I20260809 03:35:21.913030 9806 raft_consensus.cc:348] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34743 } } peers { permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 } } peers { permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 } }
I20260809 03:35:21.913298 9806 raft_consensus.cc:374] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:21.913360 9806 raft_consensus.cc:729] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6464983695c247ac838ef6adfd614cf9, State: Initialized, Role: FOLLOWER
I20260809 03:35:21.913625 9807 raft_consensus.cc:348] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34743 } } peers { permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 } } peers { permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 } }
I20260809 03:35:21.913763 9806 consensus_queue.cc:260] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34743 } } peers { permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 } } peers { permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 } }
I20260809 03:35:21.913880 9807 raft_consensus.cc:374] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:21.913985 9807 raft_consensus.cc:729] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cda3bc309d6745e08b9a07ac83c7f34a, State: Initialized, Role: FOLLOWER
I20260809 03:35:21.914098 9808 ts_tablet_manager.cc:1428] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260809 03:35:21.914506 9782 heartbeater.cc:503] Master 127.1.224.126:43699 was elected leader, sending a full tablet report...
I20260809 03:35:21.914572 9807 consensus_queue.cc:260] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34743 } } peers { permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 } } peers { permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 } }
I20260809 03:35:21.914767 9806 ts_tablet_manager.cc:1428] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260809 03:35:21.915055 9359 heartbeater.cc:503] Master 127.1.224.126:43699 was elected leader, sending a full tablet report...
I20260809 03:35:21.915266 9566 heartbeater.cc:503] Master 127.1.224.126:43699 was elected leader, sending a full tablet report...
I20260809 03:35:21.915534 9807 ts_tablet_manager.cc:1428] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:35:22.071974 9814 raft_consensus.cc:482] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:22.072517 9814 raft_consensus.cc:504] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34743 } } peers { permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 } } peers { permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 } }
I20260809 03:35:22.073525 9813 raft_consensus.cc:482] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:22.073729 9813 raft_consensus.cc:504] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34743 } } peers { permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 } } peers { permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 } }
I20260809 03:35:22.074420 9814 leader_election.cc:288] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6464983695c247ac838ef6adfd614cf9 (127.1.224.65:37019), c54e5e423496472dada8e684fdec1cb5 (127.1.224.67:37975)
I20260809 03:35:22.075446 9813 leader_election.cc:288] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cda3bc309d6745e08b9a07ac83c7f34a (127.1.224.66:34743), c54e5e423496472dada8e684fdec1cb5 (127.1.224.67:37975)
I20260809 03:35:22.079480 9568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.074375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:37019 (local address 127.1.224.66:51957)
0809 03:35:22.074777 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:22.074794 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:22.074886 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:22.075487 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:22.075490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:22.075508 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:22.075625 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:22.075632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.076700 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.076706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:22.077977 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:22.077987 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.079068 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.079072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:22.079183 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:22.079207 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:35:22.079274 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 03:35:22.079341 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310}
I20260809 03:35:22.080163 9804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.075033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:51957 (local address 127.1.224.65:37019)
0809 03:35:22.075164 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:22.075168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:22.075181 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:22.075230 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:22.075233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:22.075277 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:22.075351 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:22.075765 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.076559 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.078131 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.078821 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.078940 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:22.078996 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:22.079947 (+ 951us) server_negotiation.cc:299] Negotiation successful
0809 03:35:22.080025 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":60}
I20260809 03:35:22.080853 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.075093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:59211 (local address 127.1.224.66:34743)
0809 03:35:22.075217 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:22.075222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:22.076324 (+ 1102us) server_negotiation.cc:407] Connection header received
0809 03:35:22.076566 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:22.076572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:22.076626 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:22.076674 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:22.077103 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.077864 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.079548 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.080138 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.080229 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:22.080271 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:22.080667 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 03:35:22.080733 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":45}
I20260809 03:35:22.080848 9282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b397a5457e4aa082e7ed9a7f3043f5" candidate_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6464983695c247ac838ef6adfd614cf9" is_pre_election: true
I20260809 03:35:22.081127 9282 raft_consensus.cc:2454] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cda3bc309d6745e08b9a07ac83c7f34a in term 0.
I20260809 03:35:22.081426 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.075058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:34743 (local address 127.1.224.65:59211)
0809 03:35:22.076232 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:22.076249 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:22.076406 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:22.076822 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:22.076824 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:22.076843 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:22.076972 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:22.076978 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.078002 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.078005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:22.079395 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:22.079405 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.080366 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.080369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:22.080523 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:22.080551 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 03:35:22.081199 (+ 648us) client_negotiation.cc:240] Negotiation successful
0809 03:35:22.081273 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"mutex_wait_us":42,"thread_start_us":122,"threads_started":1}
I20260809 03:35:22.081416 9803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.074790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:49265 (local address 127.1.224.67:37975)
0809 03:35:22.074933 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:22.074940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:22.076970 (+ 2030us) server_negotiation.cc:407] Connection header received
0809 03:35:22.077160 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:22.077163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:22.077207 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:22.077254 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:22.077629 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.078322 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.079745 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.080427 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.080570 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:22.080640 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:22.081083 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 03:35:22.081230 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":59}
I20260809 03:35:22.081471 9815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.075681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:37975 (local address 127.1.224.65:42969)
0809 03:35:22.076236 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:22.076250 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:22.076345 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:22.077183 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:22.077185 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:22.077202 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:22.077298 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:22.077304 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.078419 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.078423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:22.079549 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:22.079557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.081119 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.081123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:22.081230 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:22.081250 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 03:35:22.081298 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 03:35:22.081350 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":95,"threads_started":1}
I20260809 03:35:22.081911 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.076531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:42969 (local address 127.1.224.67:37975)
0809 03:35:22.076901 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:22.076909 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:22.076924 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:35:22.076992 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:22.076995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:22.077055 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:22.077403 (+ 348us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:22.077508 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.078290 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.079686 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.080958 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.081599 (+ 641us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:22.081651 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:22.081689 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 03:35:22.081754 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":297,"thread_start_us":120,"threads_started":1}
I20260809 03:35:22.081971 9818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.074390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:37975 (local address 127.1.224.66:49265)
0809 03:35:22.076888 (+ 2498us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:22.076902 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:22.077035 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:22.077381 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:22.077383 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:22.077399 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:22.077499 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:22.077504 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.078453 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.078456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:22.079599 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:22.079611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.080706 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.080711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:22.080834 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:22.080893 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 03:35:22.081631 (+ 738us) client_negotiation.cc:240] Negotiation successful
0809 03:35:22.081734 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2437,"mutex_wait_us":1491,"thread_start_us":96,"threads_started":1}
I20260809 03:35:22.082134 9387 leader_election.cc:302] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6464983695c247ac838ef6adfd614cf9, cda3bc309d6745e08b9a07ac83c7f34a; no voters:
I20260809 03:35:22.082325 9493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b397a5457e4aa082e7ed9a7f3043f5" candidate_uuid: "6464983695c247ac838ef6adfd614cf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" is_pre_election: true
I20260809 03:35:22.082471 9712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b397a5457e4aa082e7ed9a7f3043f5" candidate_uuid: "6464983695c247ac838ef6adfd614cf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c54e5e423496472dada8e684fdec1cb5" is_pre_election: true
I20260809 03:35:22.082535 9493 raft_consensus.cc:2454] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6464983695c247ac838ef6adfd614cf9 in term 0.
I20260809 03:35:22.082609 9711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b397a5457e4aa082e7ed9a7f3043f5" candidate_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c54e5e423496472dada8e684fdec1cb5" is_pre_election: true
I20260809 03:35:22.082677 9814 raft_consensus.cc:2781] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:22.082764 9814 raft_consensus.cc:482] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:22.082751 9712 raft_consensus.cc:2454] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6464983695c247ac838ef6adfd614cf9 in term 0.
I20260809 03:35:22.082813 9814 raft_consensus.cc:3037] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:22.082960 9184 leader_election.cc:302] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6464983695c247ac838ef6adfd614cf9, cda3bc309d6745e08b9a07ac83c7f34a; no voters:
I20260809 03:35:22.083212 9813 raft_consensus.cc:2781] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:22.083299 9813 raft_consensus.cc:482] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:22.083345 9813 raft_consensus.cc:3037] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:22.084380 9814 raft_consensus.cc:504] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34743 } } peers { permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 } } peers { permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 } }
I20260809 03:35:22.084728 9813 raft_consensus.cc:504] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34743 } } peers { permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 } } peers { permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 } }
I20260809 03:35:22.084909 9814 leader_election.cc:288] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [CANDIDATE]: Term 1 election: Requested vote from peers 6464983695c247ac838ef6adfd614cf9 (127.1.224.65:37019), c54e5e423496472dada8e684fdec1cb5 (127.1.224.67:37975)
I20260809 03:35:22.085139 9813 leader_election.cc:288] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [CANDIDATE]: Term 1 election: Requested vote from peers cda3bc309d6745e08b9a07ac83c7f34a (127.1.224.66:34743), c54e5e423496472dada8e684fdec1cb5 (127.1.224.67:37975)
I20260809 03:35:22.085244 9711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b397a5457e4aa082e7ed9a7f3043f5" candidate_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c54e5e423496472dada8e684fdec1cb5"
I20260809 03:35:22.085413 9711 raft_consensus.cc:3037] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:22.085466 9712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b397a5457e4aa082e7ed9a7f3043f5" candidate_uuid: "6464983695c247ac838ef6adfd614cf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c54e5e423496472dada8e684fdec1cb5"
I20260809 03:35:22.085639 9493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b397a5457e4aa082e7ed9a7f3043f5" candidate_uuid: "6464983695c247ac838ef6adfd614cf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cda3bc309d6745e08b9a07ac83c7f34a"
I20260809 03:35:22.085855 9493 raft_consensus.cc:2379] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6464983695c247ac838ef6adfd614cf9 in current term 1: Already voted for candidate cda3bc309d6745e08b9a07ac83c7f34a in this term.
I20260809 03:35:22.086258 9184 leader_election.cc:302] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6464983695c247ac838ef6adfd614cf9; no voters: c54e5e423496472dada8e684fdec1cb5, cda3bc309d6745e08b9a07ac83c7f34a
I20260809 03:35:22.086452 9813 raft_consensus.cc:2726] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 03:35:22.086802 9711 raft_consensus.cc:2454] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cda3bc309d6745e08b9a07ac83c7f34a in term 1.
W20260809 03:35:22.085249 9783 tablet.cc:2369] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:22.087101 9391 leader_election.cc:302] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c54e5e423496472dada8e684fdec1cb5, cda3bc309d6745e08b9a07ac83c7f34a; no voters:
I20260809 03:35:22.087261 9814 raft_consensus.cc:2781] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:22.087491 9814 raft_consensus.cc:686] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 1 LEADER]: Becoming Leader. State: Replica: cda3bc309d6745e08b9a07ac83c7f34a, State: Running, Role: LEADER
I20260809 03:35:22.087743 9282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b397a5457e4aa082e7ed9a7f3043f5" candidate_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6464983695c247ac838ef6adfd614cf9"
I20260809 03:35:22.087836 9814 consensus_queue.cc:237] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34743 } } peers { permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 } } peers { permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 } }
I20260809 03:35:22.087925 9282 raft_consensus.cc:2379] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cda3bc309d6745e08b9a07ac83c7f34a in current term 1: Already voted for candidate 6464983695c247ac838ef6adfd614cf9 in this term.
I20260809 03:35:22.091327 9061 catalog_manager.cc:5495] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a reported cstate change: term changed from 0 to 1, leader changed from <none> to cda3bc309d6745e08b9a07ac83c7f34a (127.1.224.66). New cstate: current_term: 1 leader_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda3bc309d6745e08b9a07ac83c7f34a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 34743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 } health_report { overall_health: UNKNOWN } } }
W20260809 03:35:22.099718 9567 tablet.cc:2369] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:35:22.120968 9360 tablet.cc:2369] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:22.128355 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.124658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51416 (local address 127.1.224.66:34743)
0809 03:35:22.124791 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:22.124796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:22.124812 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:35:22.124914 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:22.124917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:22.124975 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:22.125091 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:22.125375 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.125896 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.126790 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:22.127014 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:22.127127 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:22.127459 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:22.127916 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:22.128058 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:22.128125 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 03:35:22.128180 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:35:22.135468 9282 raft_consensus.cc:1261] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9 [term 1 FOLLOWER]: Refusing update from remote peer cda3bc309d6745e08b9a07ac83c7f34a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:22.135463 9711 raft_consensus.cc:1261] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5 [term 1 FOLLOWER]: Refusing update from remote peer cda3bc309d6745e08b9a07ac83c7f34a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:22.136114 9820 consensus_queue.cc:1035] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6464983695c247ac838ef6adfd614cf9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:22.136368 9814 consensus_queue.cc:1035] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c54e5e423496472dada8e684fdec1cb5" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:22.145553 9826 mvcc.cc:205] Tried to move back new op lower bound from 7316465754659581952 to 7316465754475065344. Current Snapshot: MvccSnapshot[applied={T|T < 7316465754659581952}]
I20260809 03:35:22.153116 9062 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:22.153281 9062 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:22.155900 9062 catalog_manager.cc:5782] T 00000000000000000000000000000000 P caf32035b2c94de884b38385162827ee: Sending DeleteTablet for 3 replicas of tablet b1b397a5457e4aa082e7ed9a7f3043f5
I20260809 03:35:22.156790 9683 tablet_service.cc:1514] Processing DeleteTablet for tablet b1b397a5457e4aa082e7ed9a7f3043f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:22 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:42492
I20260809 03:35:22.156788 9460 tablet_service.cc:1514] Processing DeleteTablet for tablet b1b397a5457e4aa082e7ed9a7f3043f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:22 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:51402
I20260809 03:35:22.156847 9249 tablet_service.cc:1514] Processing DeleteTablet for tablet b1b397a5457e4aa082e7ed9a7f3043f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:22 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:39910
I20260809 03:35:22.157333 9835 tablet_replica.cc:330] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5: stopping tablet replica
I20260809 03:35:22.157342 9837 tablet_replica.cc:330] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a: stopping tablet replica
I20260809 03:35:22.157531 9838 tablet_replica.cc:330] T b1b397a5457e4aa082e7ed9a7f3043f5 P 6464983695c247ac838ef6adfd614cf9: stopping tablet replica
I20260809 03:35:22.157675 9837 raft_consensus.cc:2229] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:22.157742 9835 raft_consensus.cc:2229] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:22.158025 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 9161
I20260809 03:35:22.158152 9835 raft_consensus.cc:2258] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:22.158183 9837 raft_consensus.cc:2258] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:22.162276 9837 ts_tablet_manager.cc:1905] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:22.165202 9837 ts_tablet_manager.cc:1918] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:22.165356 9837 log.cc:1196] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/b1b397a5457e4aa082e7ed9a7f3043f5
I20260809 03:35:22.165728 9837 ts_tablet_manager.cc:1939] T b1b397a5457e4aa082e7ed9a7f3043f5 P cda3bc309d6745e08b9a07ac83c7f34a: Deleting consensus metadata
I20260809 03:35:22.167024 9038 catalog_manager.cc:4841] TS cda3bc309d6745e08b9a07ac83c7f34a (127.1.224.66:34743): tablet b1b397a5457e4aa082e7ed9a7f3043f5 (table test-table [id=dcec99db94c84d619993e6c78ed28f67]) successfully deleted
I20260809 03:35:22.169924 9835 ts_tablet_manager.cc:1905] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:22.174055 9835 ts_tablet_manager.cc:1918] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:22.174221 9835 log.cc:1196] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/b1b397a5457e4aa082e7ed9a7f3043f5
W20260809 03:35:22.174429 9038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:37019 (error 108)
I20260809 03:35:22.174628 9835 ts_tablet_manager.cc:1939] T b1b397a5457e4aa082e7ed9a7f3043f5 P c54e5e423496472dada8e684fdec1cb5: Deleting consensus metadata
I20260809 03:35:22.174713 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 9365
I20260809 03:35:22.188437 9040 catalog_manager.cc:4841] TS c54e5e423496472dada8e684fdec1cb5 (127.1.224.67:37975): tablet b1b397a5457e4aa082e7ed9a7f3043f5 (table test-table [id=dcec99db94c84d619993e6c78ed28f67]) successfully deleted
I20260809 03:35:22.188898 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.188506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:37019 (local address 127.0.0.1:39922)
0809 03:35:22.188673 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:22.188779 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:37019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 03:35:22.189129 9038 catalog_manager.cc:4584] TS 6464983695c247ac838ef6adfd614cf9 (127.1.224.65:37019): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1b397a5457e4aa082e7ed9a7f3043f5: Network error: Client connection negotiation failed: client connection to 127.1.224.65:37019: connect: Connection refused (error 111)
I20260809 03:35:22.194224 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 9571
I20260809 03:35:22.205832 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 9011
2026-08-09T03:35:22Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2741 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T03:35:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:22Z Disabled control of system clock
I20260809 03:35:22.246174 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:43631
--webserver_interface=127.1.224.126
--webserver_port=0
--builtin_ntp_servers=127.1.224.84:45497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:43631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 03:35:22.363480 9845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:22.363785 9845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:22.363826 9845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:22.368036 9845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:22.368119 9845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:22.368136 9845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:22.368155 9845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:22.368170 9845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:22.373493 9845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:45497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:43631
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:43631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:22.374773 9845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:22.376323 9845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:22.385655 9858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:22.385667 9852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:22.385725 9845 server_base.cc:1034] running on GCE node
W20260809 03:35:22.385681 9856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:22.386581 9845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:22.387754 9845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:22.388980 9845 hybrid_clock.cc:648] HybridClock initialized: now 1786246522388950 us; error 42 us; skew 500 ppm
I20260809 03:35:22.391079 9845 webserver.cc:460] Webserver started at http://127.1.224.126:34837/ using document root <none> and password file <none>
I20260809 03:35:22.391718 9845 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:22.391819 9845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:22.392081 9845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:22.394357 9845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "7686dc07e09c45cebf07cb60391cfdf1"
format_stamp: "Formatted at 2026-08-09 03:35:22 on dist-test-slave-jq4t"
server_key: "571e0d8494993dcdd460621e4d165010"
server_key_iv: "74b99d5c471ea22400a70f119646a298"
server_key_version: "encryptionkey@0"
I20260809 03:35:22.394996 9845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "7686dc07e09c45cebf07cb60391cfdf1"
format_stamp: "Formatted at 2026-08-09 03:35:22 on dist-test-slave-jq4t"
server_key: "571e0d8494993dcdd460621e4d165010"
server_key_iv: "74b99d5c471ea22400a70f119646a298"
server_key_version: "encryptionkey@0"
I20260809 03:35:22.399013 9845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:22.401692 9867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:22.403224 9845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:22.403404 9845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "7686dc07e09c45cebf07cb60391cfdf1"
format_stamp: "Formatted at 2026-08-09 03:35:22 on dist-test-slave-jq4t"
server_key: "571e0d8494993dcdd460621e4d165010"
server_key_iv: "74b99d5c471ea22400a70f119646a298"
server_key_version: "encryptionkey@0"
I20260809 03:35:22.403551 9845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:22.408399 9845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:22.409061 9845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:22.409255 9845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:22.416175 9845 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:43631
I20260809 03:35:22.416182 9945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:43631 every 8 connection(s)
I20260809 03:35:22.417177 9845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:22.420763 9947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:22.422538 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 9845
I20260809 03:35:22.422659 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:22.422984 1921 external_mini_cluster.cc:1428] Setting key 7d3427aebeb317e7fe4a4834673c7a3a
I20260809 03:35:22.427889 9947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1: Bootstrap starting.
I20260809 03:35:22.429145 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.424699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.1.224.126:43631)
0809 03:35:22.425399 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:22.425415 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:22.425452 (+ 37us) server_negotiation.cc:407] Connection header received
0809 03:35:22.426122 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:22.426141 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:22.426509 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:22.426872 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:22.427776 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:22.427782 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:22.427793 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:22.427805 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:22.427863 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:22.427880 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:22.428043 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:22.428116 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 03:35:22.428368 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":146,"threads_started":1}
I20260809 03:35:22.430634 9947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:22.431630 9947 log.cc:824] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:22.434132 9947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1: No bootstrap required, opened a new log
I20260809 03:35:22.438329 9947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7686dc07e09c45cebf07cb60391cfdf1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43631 } }
I20260809 03:35:22.438571 9947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:22.438616 9947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7686dc07e09c45cebf07cb60391cfdf1, State: Initialized, Role: FOLLOWER
I20260809 03:35:22.439065 9947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7686dc07e09c45cebf07cb60391cfdf1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43631 } }
I20260809 03:35:22.439254 9947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:22.439333 9947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:22.439491 9947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:22.440702 9947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7686dc07e09c45cebf07cb60391cfdf1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43631 } }
I20260809 03:35:22.441032 9947 leader_election.cc:302] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7686dc07e09c45cebf07cb60391cfdf1; no voters:
I20260809 03:35:22.441309 9947 leader_election.cc:288] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:22.441459 9959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:22.441725 9959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [term 1 LEADER]: Becoming Leader. State: Replica: 7686dc07e09c45cebf07cb60391cfdf1, State: Running, Role: LEADER
I20260809 03:35:22.442076 9959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7686dc07e09c45cebf07cb60391cfdf1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43631 } }
I20260809 03:35:22.442734 9947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:22.443740 9960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7686dc07e09c45cebf07cb60391cfdf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7686dc07e09c45cebf07cb60391cfdf1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43631 } } }
I20260809 03:35:22.443878 9960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:22.444295 9964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7686dc07e09c45cebf07cb60391cfdf1. Latest consensus state: current_term: 1 leader_uuid: "7686dc07e09c45cebf07cb60391cfdf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7686dc07e09c45cebf07cb60391cfdf1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43631 } } }
I20260809 03:35:22.444396 9964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:22.444707 9978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:22.447504 9978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:22.453009 9978 catalog_manager.cc:1347] Generated new cluster ID: 8b689821c92e4fc79aeb981978b3001e
I20260809 03:35:22.453085 9978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:22.470595 9978 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:22.471055 9978 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:22.483220 9978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1: Generated new TSK 0
I20260809 03:35:22.483999 9978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:35:22.496605 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43631
--builtin_ntp_servers=127.1.224.84:45497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 03:35:22.620416 9990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:22.620704 9990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:22.620744 9990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:22.624755 9990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:22.624809 9990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:22.624863 9990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:22.629491 9990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:45497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:22.630549 9990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:22.632005 9990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:22.642480 9998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:22.642462 9996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:22.642462 10000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:22.642823 9990 server_base.cc:1034] running on GCE node
I20260809 03:35:22.643338 9990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:22.644199 9990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:22.645406 9990 hybrid_clock.cc:648] HybridClock initialized: now 1786246522645377 us; error 37 us; skew 500 ppm
I20260809 03:35:22.647471 9990 webserver.cc:460] Webserver started at http://127.1.224.65:36855/ using document root <none> and password file <none>
I20260809 03:35:22.648098 9990 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:22.648165 9990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:22.648362 9990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:22.650692 9990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "2a6794fde9694371ad13491438212f6e"
format_stamp: "Formatted at 2026-08-09 03:35:22 on dist-test-slave-jq4t"
server_key: "bb8a6b2222ceaafaa4483745d3e730e4"
server_key_iv: "bf47306f3f878b6d3f3518a6b2889b8f"
server_key_version: "encryptionkey@0"
I20260809 03:35:22.651295 9990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "2a6794fde9694371ad13491438212f6e"
format_stamp: "Formatted at 2026-08-09 03:35:22 on dist-test-slave-jq4t"
server_key: "bb8a6b2222ceaafaa4483745d3e730e4"
server_key_iv: "bf47306f3f878b6d3f3518a6b2889b8f"
server_key_version: "encryptionkey@0"
I20260809 03:35:22.655651 9990 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 03:35:22.658596 10010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:22.660182 9990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:22.660450 9990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "2a6794fde9694371ad13491438212f6e"
format_stamp: "Formatted at 2026-08-09 03:35:22 on dist-test-slave-jq4t"
server_key: "bb8a6b2222ceaafaa4483745d3e730e4"
server_key_iv: "bf47306f3f878b6d3f3518a6b2889b8f"
server_key_version: "encryptionkey@0"
I20260809 03:35:22.660686 9990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:22.666376 9990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:22.667133 9990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:22.667363 9990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:22.668033 9990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:22.669231 9990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:22.669312 9990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:22.669394 9990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:22.669436 9990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:22.679047 9990 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:39127
I20260809 03:35:22.679068 10181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:39127 every 8 connection(s)
I20260809 03:35:22.680115 9990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:22.684613 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 9990
I20260809 03:35:22.684727 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:22.685048 1921 external_mini_cluster.cc:1428] Setting key 91a0410808e480d08e621d6ff9cd1ace
I20260809 03:35:22.687326 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43631
--builtin_ntp_servers=127.1.224.84:45497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 03:35:22.687347 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.682384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:40629 (local address 127.1.224.126:43631)
0809 03:35:22.682537 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:22.682541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:22.683683 (+ 1142us) server_negotiation.cc:407] Connection header received
0809 03:35:22.685407 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:22.685411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:22.685465 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:22.685557 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:22.686852 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:22.686858 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:22.686865 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:22.686879 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:22.686901 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:22.686909 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:22.687008 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:22.687163 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 03:35:22.687210 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:35:22.688271 10184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.682789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:43631 (local address 127.1.224.65:40629)
0809 03:35:22.683518 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:22.683574 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:22.685146 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:22.685792 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:22.685799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:22.685894 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:22.686569 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:22.686571 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:22.686616 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:22.686692 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:22.686996 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:22.687007 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:22.687013 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:35:22.687214 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 03:35:22.687483 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":142,"threads_started":1}
I20260809 03:35:22.689762 10182 heartbeater.cc:347] Connected to a master server at 127.1.224.126:43631
I20260809 03:35:22.690038 10182 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:22.690478 10182 heartbeater.cc:511] Master 127.1.224.126:43631 requested a full tablet report, sending...
I20260809 03:35:22.692026 9889 ts_manager.cc:194] Registered new tserver with Master: 2a6794fde9694371ad13491438212f6e (127.1.224.65:39127)
W20260809 03:35:22.809262 10185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:22.809609 10185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:22.809705 10185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:22.813887 10185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:22.813982 10185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:22.814076 10185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:22.819161 10185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:45497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:22.820544 10185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:22.822135 10185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:22.832995 10197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:22.833057 10194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:22.833237 10185 server_base.cc:1034] running on GCE node
W20260809 03:35:22.833002 10195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:22.833904 10185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:22.834686 10185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:22.835901 10185 hybrid_clock.cc:648] HybridClock initialized: now 1786246522835881 us; error 36 us; skew 500 ppm
I20260809 03:35:22.838130 10185 webserver.cc:460] Webserver started at http://127.1.224.66:46023/ using document root <none> and password file <none>
I20260809 03:35:22.838788 10185 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:22.838866 10185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:22.839103 10185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:22.841447 10185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "617238c491494bf794e4a3c06ccdaf71"
format_stamp: "Formatted at 2026-08-09 03:35:22 on dist-test-slave-jq4t"
server_key: "aa485de1d886bccfca96849f1cfa70fd"
server_key_iv: "6548f871863838f7b2085745807157dc"
server_key_version: "encryptionkey@0"
I20260809 03:35:22.842051 10185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "617238c491494bf794e4a3c06ccdaf71"
format_stamp: "Formatted at 2026-08-09 03:35:22 on dist-test-slave-jq4t"
server_key: "aa485de1d886bccfca96849f1cfa70fd"
server_key_iv: "6548f871863838f7b2085745807157dc"
server_key_version: "encryptionkey@0"
I20260809 03:35:22.846302 10185 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:22.848883 10207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:22.850225 10185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:22.850473 10185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "617238c491494bf794e4a3c06ccdaf71"
format_stamp: "Formatted at 2026-08-09 03:35:22 on dist-test-slave-jq4t"
server_key: "aa485de1d886bccfca96849f1cfa70fd"
server_key_iv: "6548f871863838f7b2085745807157dc"
server_key_version: "encryptionkey@0"
I20260809 03:35:22.850605 10185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:22.855413 10185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:22.856029 10185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:22.856222 10185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:22.856859 10185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:22.857797 10185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:22.857872 10185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:22.857944 10185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:22.857985 10185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:22.866945 10185 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:33215
I20260809 03:35:22.866962 10365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:33215 every 8 connection(s)
I20260809 03:35:22.867946 10185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:22.874331 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.869900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:40411 (local address 127.1.224.126:43631)
0809 03:35:22.870046 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:22.870050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:22.871115 (+ 1065us) server_negotiation.cc:407] Connection header received
0809 03:35:22.872464 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:22.872468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:22.872562 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:22.872672 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:22.873818 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:22.873821 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:22.873823 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:22.873836 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:22.873857 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:22.873865 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:22.873967 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:22.874128 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 03:35:22.874180 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:35:22.875241 10369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:22.870273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:43631 (local address 127.1.224.66:40411)
0809 03:35:22.870954 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:22.871001 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:22.872237 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:22.872847 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:22.872854 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:22.872941 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:22.873534 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:22.873536 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:22.873576 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:22.873644 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:22.873958 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:22.873967 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:22.873975 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 03:35:22.874171 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 03:35:22.874417 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":127,"threads_started":1}
I20260809 03:35:22.875595 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 10185
I20260809 03:35:22.875741 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:22.876122 1921 external_mini_cluster.cc:1428] Setting key 806277cbf2ac96e5e0bcaeb536d05ad7
I20260809 03:35:22.876710 10367 heartbeater.cc:347] Connected to a master server at 127.1.224.126:43631
I20260809 03:35:22.876993 10367 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:22.877444 10367 heartbeater.cc:511] Master 127.1.224.126:43631 requested a full tablet report, sending...
I20260809 03:35:22.878355 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43631
--builtin_ntp_servers=127.1.224.84:45497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 03:35:22.878654 9889 ts_manager.cc:194] Registered new tserver with Master: 617238c491494bf794e4a3c06ccdaf71 (127.1.224.66:33215)
W20260809 03:35:22.998724 10370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:22.999085 10370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:22.999163 10370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:23.003510 10370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:23.003639 10370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:23.003744 10370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:23.009053 10370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:45497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:23.010375 10370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:23.011976 10370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:23.022648 10378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:23.022630 10377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:23.022773 10370 server_base.cc:1034] running on GCE node
W20260809 03:35:23.022630 10380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:23.023474 10370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:23.024188 10370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:23.025385 10370 hybrid_clock.cc:648] HybridClock initialized: now 1786246523025337 us; error 61 us; skew 500 ppm
I20260809 03:35:23.027348 10370 webserver.cc:460] Webserver started at http://127.1.224.67:46639/ using document root <none> and password file <none>
I20260809 03:35:23.027990 10370 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:23.028088 10370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:23.028333 10370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:23.030736 10370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "c66349b3a8f242b3ac33f0a712491507"
format_stamp: "Formatted at 2026-08-09 03:35:23 on dist-test-slave-jq4t"
server_key: "acc4ebb50272941bb8679f329298cb61"
server_key_iv: "ac7f7f0ddc406fb9ac14893642def2f4"
server_key_version: "encryptionkey@0"
I20260809 03:35:23.031355 10370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "c66349b3a8f242b3ac33f0a712491507"
format_stamp: "Formatted at 2026-08-09 03:35:23 on dist-test-slave-jq4t"
server_key: "acc4ebb50272941bb8679f329298cb61"
server_key_iv: "ac7f7f0ddc406fb9ac14893642def2f4"
server_key_version: "encryptionkey@0"
I20260809 03:35:23.035375 10370 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:35:23.038033 10389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:23.039419 10370 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 03:35:23.039566 10370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "c66349b3a8f242b3ac33f0a712491507"
format_stamp: "Formatted at 2026-08-09 03:35:23 on dist-test-slave-jq4t"
server_key: "acc4ebb50272941bb8679f329298cb61"
server_key_iv: "ac7f7f0ddc406fb9ac14893642def2f4"
server_key_version: "encryptionkey@0"
I20260809 03:35:23.039708 10370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:23.044831 10370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:23.045485 10370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:23.045677 10370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:23.046233 10370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:23.047176 10370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:23.047245 10370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:23.047315 10370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:23.047367 10370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:23.056550 10370 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:36121
I20260809 03:35:23.056586 10562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:36121 every 8 connection(s)
I20260809 03:35:23.057523 10370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:23.063832 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.059455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:40307 (local address 127.1.224.126:43631)
0809 03:35:23.059605 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:23.059609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:23.060684 (+ 1075us) server_negotiation.cc:407] Connection header received
0809 03:35:23.062053 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:23.062057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:23.062109 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:23.062219 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:23.063352 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:23.063356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:23.063359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:23.063373 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:23.063404 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:23.063413 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:23.063525 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:23.063632 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 03:35:23.063682 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:35:23.064881 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.059801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:43631 (local address 127.1.224.67:40307)
0809 03:35:23.060459 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:23.060561 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:23.061821 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:23.062403 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:23.062409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:23.062502 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:23.063076 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:23.063077 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:23.063121 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:23.063191 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:23.063496 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:23.063505 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:23.063510 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:35:23.063681 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 03:35:23.063935 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":123,"threads_started":1}
I20260809 03:35:23.066139 10563 heartbeater.cc:347] Connected to a master server at 127.1.224.126:43631
I20260809 03:35:23.066458 10563 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:23.066491 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 10370
I20260809 03:35:23.066615 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:23.066903 1921 external_mini_cluster.cc:1428] Setting key 86eec19f2858be31924db518b8b2e14b
I20260809 03:35:23.066999 10563 heartbeater.cc:511] Master 127.1.224.126:43631 requested a full tablet report, sending...
I20260809 03:35:23.068176 9889 ts_manager.cc:194] Registered new tserver with Master: c66349b3a8f242b3ac33f0a712491507 (127.1.224.67:36121)
I20260809 03:35:23.069109 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:35:23.072602 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.071049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56728 (local address 127.1.224.126:43631)
0809 03:35:23.071184 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:23.071188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:23.071334 (+ 146us) server_negotiation.cc:407] Connection header received
0809 03:35:23.071440 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:23.071442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:23.071470 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:23.071542 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:23.072230 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:23.072233 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:23.072235 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:23.072246 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:23.072264 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:23.072272 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:23.072340 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:23.072402 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 03:35:23.072432 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:35:23.075438 9888 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:23.078672 9888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:23.090842 10585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.087703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39127 (local address 127.0.0.1:52576)
0809 03:35:23.088560 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:23.088575 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:23.088668 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:23.089576 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:23.089579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:23.089596 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:23.090023 (+ 427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:23.090025 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:23.090037 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:23.090112 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:23.090467 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:23.090476 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:23.090482 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:35:23.090604 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 03:35:23.090680 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":90,"threads_started":1}
I20260809 03:35:23.090914 10580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.087649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36672 (local address 127.1.224.66:33215)
0809 03:35:23.088132 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:23.088146 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:23.088175 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:35:23.088238 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:23.088244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:23.088446 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:23.088637 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:23.090289 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:23.090294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:23.090304 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:23.090317 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:23.090357 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:23.090389 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:23.090485 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:23.090668 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 03:35:23.090739 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":146,"threads_started":1}
I20260809 03:35:23.090948 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.088308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52576 (local address 127.1.224.65:39127)
0809 03:35:23.089151 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:23.089163 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:23.089188 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:35:23.089245 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:23.089253 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:23.089429 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:23.089586 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:23.090218 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:23.090223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:23.090235 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:23.090247 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:23.090287 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:23.090315 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:23.090419 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:23.090633 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 03:35:23.090724 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":101,"threads_started":1}
I20260809 03:35:23.091264 10577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.087484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:33215 (local address 127.0.0.1:36672)
0809 03:35:23.087970 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:23.088030 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:23.088139 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:23.089021 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:23.089030 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:23.089115 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:23.090012 (+ 897us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:23.090014 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:23.090037 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:23.090110 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:23.090536 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:23.090541 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:23.090547 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:35:23.091089 (+ 542us) client_negotiation.cc:240] Negotiation successful
0809 03:35:23.091130 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":112,"threads_started":1}
I20260809 03:35:23.092147 10579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.088226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:36121 (local address 127.0.0.1:47770)
0809 03:35:23.088774 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:23.088788 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:23.088864 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:23.090320 (+ 1456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:23.090323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:23.090337 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:23.090816 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:23.090820 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:23.090848 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:23.090876 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:23.091885 (+ 1009us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:23.091888 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:23.091892 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:35:23.091992 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 03:35:23.092027 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":98,"threads_started":1}
I20260809 03:35:23.092588 10589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.088504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47770 (local address 127.1.224.67:36121)
0809 03:35:23.089530 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:23.089545 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:23.089574 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:35:23.089633 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:23.089639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:23.089813 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:23.089951 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:23.091621 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:23.091627 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:23.091638 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:23.091652 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:23.091696 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:23.091731 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:23.091835 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:23.092322 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 03:35:23.092395 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":914,"thread_start_us":132,"threads_started":1}
I20260809 03:35:23.094076 10086 tablet_service.cc:1467] Processing CreateTablet for tablet 14d56e67bd5249a2b1a254353bfd0bc5 (DEFAULT_TABLE table=test-table [id=0e497b7cf02e4ffbaf8876d72e2b8be6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:23.094116 10269 tablet_service.cc:1467] Processing CreateTablet for tablet 14d56e67bd5249a2b1a254353bfd0bc5 (DEFAULT_TABLE table=test-table [id=0e497b7cf02e4ffbaf8876d72e2b8be6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:23.095224 10269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14d56e67bd5249a2b1a254353bfd0bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:23.095229 10086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14d56e67bd5249a2b1a254353bfd0bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:23.095177 10467 tablet_service.cc:1467] Processing CreateTablet for tablet 14d56e67bd5249a2b1a254353bfd0bc5 (DEFAULT_TABLE table=test-table [id=0e497b7cf02e4ffbaf8876d72e2b8be6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:23.096132 10467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14d56e67bd5249a2b1a254353bfd0bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:23.101822 10590 tablet_bootstrap.cc:492] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e: Bootstrap starting.
I20260809 03:35:23.103550 10592 tablet_bootstrap.cc:492] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71: Bootstrap starting.
I20260809 03:35:23.104316 10590 tablet_bootstrap.cc:654] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:23.104837 10594 tablet_bootstrap.cc:492] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507: Bootstrap starting.
I20260809 03:35:23.105211 10590 log.cc:824] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:23.105940 10592 tablet_bootstrap.cc:654] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:23.106818 10592 log.cc:824] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:23.107133 10594 tablet_bootstrap.cc:654] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:23.107510 10590 tablet_bootstrap.cc:492] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e: No bootstrap required, opened a new log
I20260809 03:35:23.107748 10590 ts_tablet_manager.cc:1397] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:35:23.108050 10594 log.cc:824] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:23.109191 10592 tablet_bootstrap.cc:492] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71: No bootstrap required, opened a new log
I20260809 03:35:23.109409 10592 ts_tablet_manager.cc:1397] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.004s
I20260809 03:35:23.109819 10594 tablet_bootstrap.cc:492] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507: No bootstrap required, opened a new log
I20260809 03:35:23.110010 10594 ts_tablet_manager.cc:1397] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:35:23.112795 10590 raft_consensus.cc:348] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617238c491494bf794e4a3c06ccdaf71" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33215 } } peers { permanent_uuid: "c66349b3a8f242b3ac33f0a712491507" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36121 } } peers { permanent_uuid: "2a6794fde9694371ad13491438212f6e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39127 } }
I20260809 03:35:23.113117 10590 raft_consensus.cc:374] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:23.113173 10590 raft_consensus.cc:729] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6794fde9694371ad13491438212f6e, State: Initialized, Role: FOLLOWER
I20260809 03:35:23.113606 10594 raft_consensus.cc:348] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617238c491494bf794e4a3c06ccdaf71" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33215 } } peers { permanent_uuid: "c66349b3a8f242b3ac33f0a712491507" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36121 } } peers { permanent_uuid: "2a6794fde9694371ad13491438212f6e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39127 } }
I20260809 03:35:23.113744 10590 consensus_queue.cc:260] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617238c491494bf794e4a3c06ccdaf71" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33215 } } peers { permanent_uuid: "c66349b3a8f242b3ac33f0a712491507" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36121 } } peers { permanent_uuid: "2a6794fde9694371ad13491438212f6e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39127 } }
I20260809 03:35:23.113869 10594 raft_consensus.cc:374] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:23.113932 10594 raft_consensus.cc:729] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c66349b3a8f242b3ac33f0a712491507, State: Initialized, Role: FOLLOWER
I20260809 03:35:23.114146 10592 raft_consensus.cc:348] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617238c491494bf794e4a3c06ccdaf71" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33215 } } peers { permanent_uuid: "c66349b3a8f242b3ac33f0a712491507" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36121 } } peers { permanent_uuid: "2a6794fde9694371ad13491438212f6e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39127 } }
I20260809 03:35:23.114408 10592 raft_consensus.cc:374] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:23.114347 10594 consensus_queue.cc:260] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617238c491494bf794e4a3c06ccdaf71" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33215 } } peers { permanent_uuid: "c66349b3a8f242b3ac33f0a712491507" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36121 } } peers { permanent_uuid: "2a6794fde9694371ad13491438212f6e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39127 } }
I20260809 03:35:23.114464 10592 raft_consensus.cc:729] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 617238c491494bf794e4a3c06ccdaf71, State: Initialized, Role: FOLLOWER
I20260809 03:35:23.114728 10182 heartbeater.cc:503] Master 127.1.224.126:43631 was elected leader, sending a full tablet report...
I20260809 03:35:23.114861 10592 consensus_queue.cc:260] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617238c491494bf794e4a3c06ccdaf71" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33215 } } peers { permanent_uuid: "c66349b3a8f242b3ac33f0a712491507" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36121 } } peers { permanent_uuid: "2a6794fde9694371ad13491438212f6e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39127 } }
I20260809 03:35:23.115008 10590 ts_tablet_manager.cc:1428] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 03:35:23.115875 10592 ts_tablet_manager.cc:1428] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260809 03:35:23.115942 10367 heartbeater.cc:503] Master 127.1.224.126:43631 was elected leader, sending a full tablet report...
I20260809 03:35:23.115988 10594 ts_tablet_manager.cc:1428] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:35:23.116499 10563 heartbeater.cc:503] Master 127.1.224.126:43631 was elected leader, sending a full tablet report...
W20260809 03:35:23.118973 10368 tablet.cc:2369] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:35:23.181636 10183 tablet.cc:2369] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:23.184217 10600 raft_consensus.cc:482] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:23.184412 10600 raft_consensus.cc:504] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617238c491494bf794e4a3c06ccdaf71" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33215 } } peers { permanent_uuid: "c66349b3a8f242b3ac33f0a712491507" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36121 } } peers { permanent_uuid: "2a6794fde9694371ad13491438212f6e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39127 } }
I20260809 03:35:23.185793 10600 leader_election.cc:288] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 617238c491494bf794e4a3c06ccdaf71 (127.1.224.66:33215), c66349b3a8f242b3ac33f0a712491507 (127.1.224.67:36121)
I20260809 03:35:23.188211 10184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.186096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:33215 (local address 127.1.224.65:53355)
0809 03:35:23.186233 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:23.186251 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:23.186356 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:23.186763 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:23.186766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:23.186787 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:23.187396 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:23.187398 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:23.187409 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:23.187435 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:23.187931 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:23.187934 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:23.187938 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:35:23.188024 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 03:35:23.188064 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 03:35:23.188264 10580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.186249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:53355 (local address 127.1.224.66:33215)
0809 03:35:23.186399 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:23.186404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:23.186423 (+ 19us) server_negotiation.cc:407] Connection header received
0809 03:35:23.186481 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:23.186485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:23.186534 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:23.186662 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:23.187601 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:23.187605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:23.187607 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:23.187621 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:23.187644 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:23.187653 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:23.187736 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:23.188037 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 03:35:23.188101 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:35:23.188417 10589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.186570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:56233 (local address 127.1.224.67:36121)
0809 03:35:23.186722 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:23.186728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:23.186913 (+ 185us) server_negotiation.cc:407] Connection header received
0809 03:35:23.186972 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:23.186978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:23.187025 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:23.187091 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:23.187962 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:23.187965 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:23.187968 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:23.187981 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:23.188006 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:23.188018 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:23.188104 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:23.188251 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 03:35:23.188296 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:35:23.188819 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.186415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:36121 (local address 127.1.224.65:56233)
0809 03:35:23.186818 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:23.186831 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:23.186910 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:23.187148 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:23.187151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:23.187167 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:23.187795 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:23.187796 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:23.187804 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:23.187828 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:23.188147 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:23.188150 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:23.188154 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:35:23.188626 (+ 472us) client_negotiation.cc:240] Negotiation successful
0809 03:35:23.188670 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":135,"threads_started":1}
I20260809 03:35:23.189229 10295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d56e67bd5249a2b1a254353bfd0bc5" candidate_uuid: "2a6794fde9694371ad13491438212f6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "617238c491494bf794e4a3c06ccdaf71" is_pre_election: true
I20260809 03:35:23.189502 10495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d56e67bd5249a2b1a254353bfd0bc5" candidate_uuid: "2a6794fde9694371ad13491438212f6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c66349b3a8f242b3ac33f0a712491507" is_pre_election: true
I20260809 03:35:23.189610 10295 raft_consensus.cc:2454] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a6794fde9694371ad13491438212f6e in term 0.
I20260809 03:35:23.189810 10495 raft_consensus.cc:2454] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a6794fde9694371ad13491438212f6e in term 0.
I20260809 03:35:23.190193 10013 leader_election.cc:302] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6794fde9694371ad13491438212f6e, 617238c491494bf794e4a3c06ccdaf71; no voters:
I20260809 03:35:23.190493 10600 raft_consensus.cc:2781] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:23.190596 10600 raft_consensus.cc:482] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:23.190654 10600 raft_consensus.cc:3037] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:23.191818 10600 raft_consensus.cc:504] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617238c491494bf794e4a3c06ccdaf71" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33215 } } peers { permanent_uuid: "c66349b3a8f242b3ac33f0a712491507" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36121 } } peers { permanent_uuid: "2a6794fde9694371ad13491438212f6e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39127 } }
I20260809 03:35:23.192231 10600 leader_election.cc:288] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [CANDIDATE]: Term 1 election: Requested vote from peers 617238c491494bf794e4a3c06ccdaf71 (127.1.224.66:33215), c66349b3a8f242b3ac33f0a712491507 (127.1.224.67:36121)
I20260809 03:35:23.192699 10495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d56e67bd5249a2b1a254353bfd0bc5" candidate_uuid: "2a6794fde9694371ad13491438212f6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c66349b3a8f242b3ac33f0a712491507"
I20260809 03:35:23.192835 10495 raft_consensus.cc:3037] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:23.193884 10495 raft_consensus.cc:2454] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a6794fde9694371ad13491438212f6e in term 1.
I20260809 03:35:23.194216 10295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d56e67bd5249a2b1a254353bfd0bc5" candidate_uuid: "2a6794fde9694371ad13491438212f6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "617238c491494bf794e4a3c06ccdaf71"
I20260809 03:35:23.194248 10013 leader_election.cc:302] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6794fde9694371ad13491438212f6e, c66349b3a8f242b3ac33f0a712491507; no voters:
I20260809 03:35:23.194387 10295 raft_consensus.cc:3037] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:23.194450 10600 raft_consensus.cc:2781] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:23.194733 10600 raft_consensus.cc:686] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [term 1 LEADER]: Becoming Leader. State: Replica: 2a6794fde9694371ad13491438212f6e, State: Running, Role: LEADER
I20260809 03:35:23.195142 10600 consensus_queue.cc:237] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617238c491494bf794e4a3c06ccdaf71" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33215 } } peers { permanent_uuid: "c66349b3a8f242b3ac33f0a712491507" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36121 } } peers { permanent_uuid: "2a6794fde9694371ad13491438212f6e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39127 } }
I20260809 03:35:23.195811 10295 raft_consensus.cc:2454] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a6794fde9694371ad13491438212f6e in term 1.
I20260809 03:35:23.198894 9887 catalog_manager.cc:5495] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a6794fde9694371ad13491438212f6e (127.1.224.65). New cstate: current_term: 1 leader_uuid: "2a6794fde9694371ad13491438212f6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617238c491494bf794e4a3c06ccdaf71" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c66349b3a8f242b3ac33f0a712491507" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a6794fde9694371ad13491438212f6e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39127 } health_report { overall_health: HEALTHY } } }
I20260809 03:35:23.220167 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.218472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52590 (local address 127.1.224.65:39127)
0809 03:35:23.218654 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:23.218658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:23.218670 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:35:23.218758 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:23.218760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:23.218822 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:23.218925 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:23.219728 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:23.219733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:23.219737 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:23.219754 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:23.219775 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:23.219783 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:23.219879 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:23.219967 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 03:35:23.220014 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 03:35:23.226861 10295 raft_consensus.cc:1261] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71 [term 1 FOLLOWER]: Refusing update from remote peer 2a6794fde9694371ad13491438212f6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:23.226861 10495 raft_consensus.cc:1261] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507 [term 1 FOLLOWER]: Refusing update from remote peer 2a6794fde9694371ad13491438212f6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:23.227757 10600 consensus_queue.cc:1035] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "617238c491494bf794e4a3c06ccdaf71" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:23.227993 10606 consensus_queue.cc:1035] T 14d56e67bd5249a2b1a254353bfd0bc5 P 2a6794fde9694371ad13491438212f6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c66349b3a8f242b3ac33f0a712491507" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:23.236052 10609 mvcc.cc:205] Tried to move back new op lower bound from 7316465759128772608 to 7316465759010955264. Current Snapshot: MvccSnapshot[applied={T|T < 7316465759128772608}]
I20260809 03:35:23.238742 10615 mvcc.cc:205] Tried to move back new op lower bound from 7316465759128772608 to 7316465759010955264. Current Snapshot: MvccSnapshot[applied={T|T < 7316465759128772608}]
I20260809 03:35:23.239188 10612 mvcc.cc:205] Tried to move back new op lower bound from 7316465759128772608 to 7316465759010955264. Current Snapshot: MvccSnapshot[applied={T|T < 7316465759128772608}]
I20260809 03:35:23.246507 9888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:23.246718 9888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:23.249372 9888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7686dc07e09c45cebf07cb60391cfdf1: Sending DeleteTablet for 3 replicas of tablet 14d56e67bd5249a2b1a254353bfd0bc5
I20260809 03:35:23.250305 10086 tablet_service.cc:1514] Processing DeleteTablet for tablet 14d56e67bd5249a2b1a254353bfd0bc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:23 UTC) from {username='slave'} at 127.0.0.1:52576
I20260809 03:35:23.250304 10269 tablet_service.cc:1514] Processing DeleteTablet for tablet 14d56e67bd5249a2b1a254353bfd0bc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:23 UTC) from {username='slave'} at 127.0.0.1:36672
I20260809 03:35:23.250794 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 9990
I20260809 03:35:23.256652 10467 tablet_service.cc:1514] Processing DeleteTablet for tablet 14d56e67bd5249a2b1a254353bfd0bc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:23 UTC) from {username='slave'} at 127.0.0.1:47770
I20260809 03:35:23.256794 10628 tablet_replica.cc:330] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71: stopping tablet replica
I20260809 03:35:23.257213 10628 raft_consensus.cc:2229] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:23.257634 10628 raft_consensus.cc:2258] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:23.259824 10629 tablet_replica.cc:330] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507: stopping tablet replica
I20260809 03:35:23.260192 10629 raft_consensus.cc:2229] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:23.260579 10629 raft_consensus.cc:2258] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:23.260643 10628 ts_tablet_manager.cc:1905] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:23.262192 10629 ts_tablet_manager.cc:1905] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:23.263751 10628 ts_tablet_manager.cc:1918] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:23.263875 10628 log.cc:1196] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/14d56e67bd5249a2b1a254353bfd0bc5
I20260809 03:35:23.264267 10628 ts_tablet_manager.cc:1939] T 14d56e67bd5249a2b1a254353bfd0bc5 P 617238c491494bf794e4a3c06ccdaf71: Deleting consensus metadata
I20260809 03:35:23.265097 10629 ts_tablet_manager.cc:1918] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:23.265202 10629 log.cc:1196] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/14d56e67bd5249a2b1a254353bfd0bc5
I20260809 03:35:23.265533 10629 ts_tablet_manager.cc:1939] T 14d56e67bd5249a2b1a254353bfd0bc5 P c66349b3a8f242b3ac33f0a712491507: Deleting consensus metadata
W20260809 03:35:23.265766 9869 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:39127 (error 108)
I20260809 03:35:23.266263 9872 catalog_manager.cc:4841] TS 617238c491494bf794e4a3c06ccdaf71 (127.1.224.66:33215): tablet 14d56e67bd5249a2b1a254353bfd0bc5 (table test-table [id=0e497b7cf02e4ffbaf8876d72e2b8be6]) successfully deleted
I20260809 03:35:23.266721 9872 catalog_manager.cc:4841] TS c66349b3a8f242b3ac33f0a712491507 (127.1.224.67:36121): tablet 14d56e67bd5249a2b1a254353bfd0bc5 (table test-table [id=0e497b7cf02e4ffbaf8876d72e2b8be6]) successfully deleted
I20260809 03:35:23.266973 10579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.266638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39127 (local address 127.0.0.1:52602)
0809 03:35:23.266799 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:23.266880 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:39127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 03:35:23.267104 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 10185
W20260809 03:35:23.267159 9869 catalog_manager.cc:4584] TS 2a6794fde9694371ad13491438212f6e (127.1.224.65:39127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14d56e67bd5249a2b1a254353bfd0bc5: Network error: Client connection negotiation failed: client connection to 127.1.224.65:39127: connect: Connection refused (error 111)
I20260809 03:35:23.285167 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 10370
I20260809 03:35:23.303084 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 9845
2026-08-09T03:35:23Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1094 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 03:35:23.331312 1921 mini_oidc.cc:152] Starting JWKS server
I20260809 03:35:23.333338 1921 webserver.cc:460] Webserver started at https://127.0.0.1:44765/ using document root <none> and password file <none>
I20260809 03:35:23.333540 1921 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 03:35:23.333917 1921 webserver.cc:460] Webserver started at http://127.0.0.1:41691/ using document root <none> and password file <none>
2026-08-09T03:35:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:23Z Disabled control of system clock
I20260809 03:35:23.344609 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:46477
--webserver_interface=127.1.224.126
--webserver_port=0
--builtin_ntp_servers=127.1.224.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:46477
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44765/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/testchainca.cert with env {}
W20260809 03:35:23.464913 10642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:23.465214 10642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:23.465255 10642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:23.469306 10642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:23.469375 10642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:23.469391 10642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:23.469408 10642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:23.469424 10642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 03:35:23.469440 10642 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 03:35:23.469457 10642 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44765/jwks.json
I20260809 03:35:23.474969 10642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:46477
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:46477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44765/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:23.476089 10642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:23.477193 10642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:23.486104 10652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:23.486155 10648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:23.486338 10642 server_base.cc:1034] running on GCE node
W20260809 03:35:23.486104 10650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:23.487059 10642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:23.488173 10642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:23.489336 10642 hybrid_clock.cc:648] HybridClock initialized: now 1786246523489297 us; error 51 us; skew 500 ppm
I20260809 03:35:23.491348 10642 webserver.cc:460] Webserver started at http://127.1.224.126:39043/ using document root <none> and password file <none>
I20260809 03:35:23.491964 10642 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:23.492040 10642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:23.492267 10642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:23.494421 10642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "811008bcceeb4b5b92381a6413b6f1ce"
format_stamp: "Formatted at 2026-08-09 03:35:23 on dist-test-slave-jq4t"
server_key: "d317d7842cafaffbec2d4f2eefa87338"
server_key_iv: "b560f9ceec37d0c341fd379e9cf46e1c"
server_key_version: "encryptionkey@0"
I20260809 03:35:23.495012 10642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "811008bcceeb4b5b92381a6413b6f1ce"
format_stamp: "Formatted at 2026-08-09 03:35:23 on dist-test-slave-jq4t"
server_key: "d317d7842cafaffbec2d4f2eefa87338"
server_key_iv: "b560f9ceec37d0c341fd379e9cf46e1c"
server_key_version: "encryptionkey@0"
I20260809 03:35:23.498812 10642 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:23.501451 10660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:23.502730 10642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 03:35:23.502892 10642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "811008bcceeb4b5b92381a6413b6f1ce"
format_stamp: "Formatted at 2026-08-09 03:35:23 on dist-test-slave-jq4t"
server_key: "d317d7842cafaffbec2d4f2eefa87338"
server_key_iv: "b560f9ceec37d0c341fd379e9cf46e1c"
server_key_version: "encryptionkey@0"
I20260809 03:35:23.503018 10642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:23.527928 10642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:23.528877 10642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:23.529102 10642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:23.536917 10642 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:46477
I20260809 03:35:23.536929 10737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:46477 every 8 connection(s)
I20260809 03:35:23.537988 10642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:23.541253 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 10642
I20260809 03:35:23.541308 10739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:23.541406 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:23.541724 1921 external_mini_cluster.cc:1428] Setting key f93dfdae068585d1c6076504c5825912
I20260809 03:35:23.549351 10739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce: Bootstrap starting.
I20260809 03:35:23.551786 10739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:23.551841 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.543102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56840 (local address 127.1.224.126:46477)
0809 03:35:23.543774 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:23.543805 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:23.543846 (+ 41us) server_negotiation.cc:407] Connection header received
0809 03:35:23.544600 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:23.544621 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:23.545054 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:23.545424 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:23.545886 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:23.546876 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:23.548320 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:23.548901 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:23.549092 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:23.550069 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:23.550075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:23.550088 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:23.550488 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:23.550548 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:23.550570 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:23.550686 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:23.550806 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 03:35:23.551112 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":132,"threads_started":1}
I20260809 03:35:23.552794 10739 log.cc:824] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:23.555100 10739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce: No bootstrap required, opened a new log
I20260809 03:35:23.555109 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:35:23.559082 10739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811008bcceeb4b5b92381a6413b6f1ce" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46477 } }
I20260809 03:35:23.559314 10739 raft_consensus.cc:374] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:23.559371 10739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 811008bcceeb4b5b92381a6413b6f1ce, State: Initialized, Role: FOLLOWER
I20260809 03:35:23.559799 10739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811008bcceeb4b5b92381a6413b6f1ce" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46477 } }
I20260809 03:35:23.559921 10739 raft_consensus.cc:388] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:23.560003 10739 raft_consensus.cc:482] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:23.560127 10739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:23.561154 10739 raft_consensus.cc:504] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811008bcceeb4b5b92381a6413b6f1ce" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46477 } }
I20260809 03:35:23.561518 10739 leader_election.cc:302] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 811008bcceeb4b5b92381a6413b6f1ce; no voters:
I20260809 03:35:23.562027 10739 leader_election.cc:288] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:23.562194 10748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:23.562489 10748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [term 1 LEADER]: Becoming Leader. State: Replica: 811008bcceeb4b5b92381a6413b6f1ce, State: Running, Role: LEADER
I20260809 03:35:23.562861 10748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811008bcceeb4b5b92381a6413b6f1ce" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46477 } }
I20260809 03:35:23.563506 10739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:23.565032 10749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "811008bcceeb4b5b92381a6413b6f1ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811008bcceeb4b5b92381a6413b6f1ce" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46477 } } }
I20260809 03:35:23.565080 10750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 811008bcceeb4b5b92381a6413b6f1ce. Latest consensus state: current_term: 1 leader_uuid: "811008bcceeb4b5b92381a6413b6f1ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "811008bcceeb4b5b92381a6413b6f1ce" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46477 } } }
I20260809 03:35:23.565173 10749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:23.565200 10750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:23.565567 10763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:23.569114 10763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:23.575191 10763 catalog_manager.cc:1347] Generated new cluster ID: 32a1d760bfa344bcbcaa45ace92565d5
I20260809 03:35:23.575266 10763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:23.610898 10763 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:23.612387 10763 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:23.619848 10763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 811008bcceeb4b5b92381a6413b6f1ce: Generated new TSK 0
I20260809 03:35:23.620764 10763 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 03:35:23.656844 10787 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 03:35:23.657029 10787 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 03:35:23.657861 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.640166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56850 (local address 127.1.224.126:46477)
0809 03:35:23.640324 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:23.640331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:23.640651 (+ 320us) server_negotiation.cc:407] Connection header received
0809 03:35:23.640917 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:23.640920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:23.640972 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:23.641103 (+ 131us) server_negotiation.cc:226] Negotiated authn=JWT
0809 03:35:23.641392 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:23.641956 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:23.642884 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:23.643128 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:23.643244 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:23.643634 (+ 390us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 03:35:23.657351 (+ 13717us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44765/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 03:35:23.657695 (+ 344us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56850: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44765/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 03:35:23.657971 10744 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56850: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44765/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 03:35:23.659320 10786 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:23.640073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:46477 (local address 127.0.0.1:56850)
0809 03:35:23.640556 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:23.640574 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:23.640730 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:23.641123 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:23.641127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:23.641136 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 03:35:23.641276 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:23.641281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:23.642098 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:23.642104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:23.642757 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:23.642765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:23.643257 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:23.643261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:23.643367 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:23.643419 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 03:35:23.657624 (+ 14205us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44765/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 03:35:23.657902 (+ 278us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.224.126:46477: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44765/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":126,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.224.126:46477: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44765/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 03:35:23.660162 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 10642
2026-08-09T03:35:23Z chronyd exiting
I20260809 03:35:23.707145 1921 test_util.cc:175] -----------------------------------------------
I20260809 03:35:23.707238 1921 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786246494448707-1921-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (377 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 03:35:23.708403 1921 mini_oidc.cc:152] Starting JWKS server
I20260809 03:35:23.709347 1921 webserver.cc:460] Webserver started at https://127.0.0.1:42723/ using document root <none> and password file <none>
I20260809 03:35:23.709463 1921 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 03:35:23.709744 1921 webserver.cc:460] Webserver started at http://127.0.0.1:37063/ using document root <none> and password file <none>
2026-08-09T03:35:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:23Z Disabled control of system clock
I20260809 03:35:23.719846 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:41185
--webserver_interface=127.1.224.126
--webserver_port=0
--builtin_ntp_servers=127.1.224.84:45979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:41185
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42723/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/testchainca.cert with env {}
W20260809 03:35:23.839102 10801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:23.839389 10801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:23.839429 10801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:23.843488 10801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:23.843555 10801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:23.843571 10801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:23.843588 10801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:23.843604 10801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 03:35:23.843619 10801 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 03:35:23.843633 10801 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42723/jwks.json
I20260809 03:35:23.849295 10801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:45979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:41185
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:41185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42723/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:23.850421 10801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:23.851457 10801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:23.860635 10811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:23.860690 10809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:23.860807 10801 server_base.cc:1034] running on GCE node
W20260809 03:35:23.860690 10808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:23.861892 10801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:23.863031 10801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:23.864214 10801 hybrid_clock.cc:648] HybridClock initialized: now 1786246523864193 us; error 36 us; skew 500 ppm
I20260809 03:35:23.866420 10801 webserver.cc:460] Webserver started at http://127.1.224.126:46817/ using document root <none> and password file <none>
I20260809 03:35:23.867076 10801 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:23.867144 10801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:23.867383 10801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:23.869670 10801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "4f59331d17a4471aa34f1a30a1bd897e"
format_stamp: "Formatted at 2026-08-09 03:35:23 on dist-test-slave-jq4t"
server_key: "d1b109b20ec5630db81aaac4d386cb16"
server_key_iv: "15caa4218251af4544ec2ae26495e527"
server_key_version: "encryptionkey@0"
I20260809 03:35:23.870283 10801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "4f59331d17a4471aa34f1a30a1bd897e"
format_stamp: "Formatted at 2026-08-09 03:35:23 on dist-test-slave-jq4t"
server_key: "d1b109b20ec5630db81aaac4d386cb16"
server_key_iv: "15caa4218251af4544ec2ae26495e527"
server_key_version: "encryptionkey@0"
I20260809 03:35:23.874300 10801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:23.876959 10819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:23.878278 10801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:23.878437 10801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "4f59331d17a4471aa34f1a30a1bd897e"
format_stamp: "Formatted at 2026-08-09 03:35:23 on dist-test-slave-jq4t"
server_key: "d1b109b20ec5630db81aaac4d386cb16"
server_key_iv: "15caa4218251af4544ec2ae26495e527"
server_key_version: "encryptionkey@0"
I20260809 03:35:23.878566 10801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:23.898728 10801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:23.899571 10801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:23.899804 10801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:23.908094 10801 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:41185
I20260809 03:35:23.908113 10908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:41185 every 8 connection(s)
I20260809 03:35:23.909162 10801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:23.912741 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:23.916808 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 10801
I20260809 03:35:23.916962 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:23.917279 1921 external_mini_cluster.cc:1428] Setting key fb9b239824ef4927923080eef9ace13c
I20260809 03:35:23.919852 10909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e: Bootstrap starting.
I20260809 03:35:23.922497 10909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:23.923311 10909 log.cc:824] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:23.925695 10909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e: No bootstrap required, opened a new log
I20260809 03:35:23.927737 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.918870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44432 (local address 127.1.224.126:41185)
0809 03:35:23.919654 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:23.919671 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:23.919711 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:35:23.920650 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:23.920672 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:23.921101 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:23.921509 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:23.922001 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:23.923094 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:23.924101 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:23.924675 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:23.924849 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:23.925874 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:23.925880 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:23.925895 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:23.926312 (+ 417us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:23.926375 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:23.926396 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:23.926509 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:23.926681 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 03:35:23.926971 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":167,"threads_started":1}
I20260809 03:35:23.930168 10909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f59331d17a4471aa34f1a30a1bd897e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41185 } }
I20260809 03:35:23.930449 10909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:23.930536 10909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f59331d17a4471aa34f1a30a1bd897e, State: Initialized, Role: FOLLOWER
I20260809 03:35:23.931077 10909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f59331d17a4471aa34f1a30a1bd897e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41185 } }
I20260809 03:35:23.931187 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:35:23.931252 10909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:23.931324 10909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:23.931448 10909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:23.932696 10909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f59331d17a4471aa34f1a30a1bd897e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41185 } }
I20260809 03:35:23.933110 10909 leader_election.cc:302] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f59331d17a4471aa34f1a30a1bd897e; no voters:
I20260809 03:35:23.933501 10909 leader_election.cc:288] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:23.933634 10916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:23.933923 10916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [term 1 LEADER]: Becoming Leader. State: Replica: 4f59331d17a4471aa34f1a30a1bd897e, State: Running, Role: LEADER
I20260809 03:35:23.934242 10916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f59331d17a4471aa34f1a30a1bd897e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41185 } }
I20260809 03:35:23.934845 10909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:23.936012 10917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f59331d17a4471aa34f1a30a1bd897e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f59331d17a4471aa34f1a30a1bd897e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41185 } } }
I20260809 03:35:23.936153 10917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:23.936441 10918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f59331d17a4471aa34f1a30a1bd897e. Latest consensus state: current_term: 1 leader_uuid: "4f59331d17a4471aa34f1a30a1bd897e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f59331d17a4471aa34f1a30a1bd897e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41185 } } }
I20260809 03:35:23.936568 10918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:23.937466 10927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:23.940258 10927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:23.946281 10927 catalog_manager.cc:1347] Generated new cluster ID: 3d6d65c41c954141a48f352373fa634f
I20260809 03:35:23.946363 10927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:23.965263 10927 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:23.966555 10927 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:23.974330 10927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f59331d17a4471aa34f1a30a1bd897e: Generated new TSK 0
I20260809 03:35:23.975291 10927 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 03:35:24.001425 10950 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 03:35:24.001549 10950 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 03:35:24.002470 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:23.984786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44448 (local address 127.1.224.126:41185)
0809 03:35:23.984938 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:23.984943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:23.985118 (+ 175us) server_negotiation.cc:407] Connection header received
0809 03:35:23.985272 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:23.985276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:23.985341 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:23.985467 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0809 03:35:23.985714 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:23.986576 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:23.987497 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:23.987857 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:23.987960 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:23.988268 (+ 308us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 03:35:24.001950 (+ 13682us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 03:35:24.002298 (+ 348us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44448: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 03:35:24.002555 10913 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44448: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 03:35:24.002635 10949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:23.984677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:41185 (local address 127.0.0.1:44448)
0809 03:35:23.985026 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:23.985040 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:23.985146 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:23.985482 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:23.985485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:23.985493 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 03:35:23.985602 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:23.985607 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:23.986710 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:23.986713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:23.987362 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:23.987370 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:23.987973 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:23.987976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:23.988045 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:23.988071 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 03:35:24.002229 (+ 14158us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 03:35:24.002412 (+ 183us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.224.126:41185: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":126,"threads_started":1}
I20260809 03:35:24.003494 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 10801
2026-08-09T03:35:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (343 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 03:35:24.051754 1921 mini_oidc.cc:152] Starting JWKS server
I20260809 03:35:24.052755 1921 webserver.cc:460] Webserver started at https://127.0.0.1:36489/ using document root <none> and password file <none>
I20260809 03:35:24.052873 1921 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 03:35:24.053158 1921 webserver.cc:460] Webserver started at http://127.0.0.1:37745/ using document root <none> and password file <none>
2026-08-09T03:35:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:24Z Disabled control of system clock
I20260809 03:35:24.063921 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:42077
--webserver_interface=127.1.224.126
--webserver_port=0
--builtin_ntp_servers=127.1.224.84:45539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:42077
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36489/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 03:35:24.182775 10964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:24.183074 10964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:24.183115 10964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:24.187073 10964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:24.187139 10964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:24.187155 10964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:24.187172 10964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:24.187187 10964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 03:35:24.187204 10964 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 03:35:24.187217 10964 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36489/jwks.json
I20260809 03:35:24.192049 10964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:45539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:42077
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:42077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36489/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:24.193282 10964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:24.194773 10964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:24.203799 10970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:24.203791 10969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:24.203893 10964 server_base.cc:1034] running on GCE node
W20260809 03:35:24.203791 10972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:24.204710 10964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:24.205924 10964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:24.207129 10964 hybrid_clock.cc:648] HybridClock initialized: now 1786246524207099 us; error 45 us; skew 500 ppm
I20260809 03:35:24.209373 10964 webserver.cc:460] Webserver started at http://127.1.224.126:42925/ using document root <none> and password file <none>
I20260809 03:35:24.210042 10964 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:24.210153 10964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:24.210414 10964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:24.212798 10964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "1ae7ed0aa3984cb1bbba6a90fe4ff803"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-jq4t"
server_key: "3107cb79b80ce673b58dd56a9a255a8d"
server_key_iv: "b8bb7d20cb8bd02a2c72f65b5044d5d9"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.213436 10964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "1ae7ed0aa3984cb1bbba6a90fe4ff803"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-jq4t"
server_key: "3107cb79b80ce673b58dd56a9a255a8d"
server_key_iv: "b8bb7d20cb8bd02a2c72f65b5044d5d9"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.217573 10964 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:35:24.220371 10982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.221928 10964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:24.222092 10964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "1ae7ed0aa3984cb1bbba6a90fe4ff803"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-jq4t"
server_key: "3107cb79b80ce673b58dd56a9a255a8d"
server_key_iv: "b8bb7d20cb8bd02a2c72f65b5044d5d9"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.222236 10964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:24.252804 10964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:24.253765 10964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:24.253962 10964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:24.262074 11056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:42077 every 8 connection(s)
I20260809 03:35:24.262053 10964 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:42077
I20260809 03:35:24.263283 10964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:24.266741 11057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:24.271090 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 10964
I20260809 03:35:24.271246 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:24.271591 1921 external_mini_cluster.cc:1428] Setting key 1b2de1539226cc599fa7ff40b00f70a7
I20260809 03:35:24.273834 11057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803: Bootstrap starting.
I20260809 03:35:24.276281 11057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:24.277204 11057 log.cc:824] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:24.279286 11057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803: No bootstrap required, opened a new log
I20260809 03:35:24.282030 11060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.273259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58122 (local address 127.1.224.126:42077)
0809 03:35:24.274144 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.274162 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.274206 (+ 44us) server_negotiation.cc:407] Connection header received
0809 03:35:24.274922 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.274941 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.275368 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.275758 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:24.276258 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.277480 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.278447 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.278963 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.279146 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.280123 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:24.280128 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:24.280143 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:24.280624 (+ 481us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:24.280686 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:24.280709 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:24.280824 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.280990 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.281267 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":173,"threads_started":1}
I20260809 03:35:24.283596 11057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae7ed0aa3984cb1bbba6a90fe4ff803" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42077 } }
I20260809 03:35:24.283864 11057 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:24.283947 11057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ae7ed0aa3984cb1bbba6a90fe4ff803, State: Initialized, Role: FOLLOWER
I20260809 03:35:24.284512 11057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae7ed0aa3984cb1bbba6a90fe4ff803" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42077 } }
I20260809 03:35:24.284698 11057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:24.284773 11057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:24.284880 11057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:24.285547 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:35:24.286123 11057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae7ed0aa3984cb1bbba6a90fe4ff803" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42077 } }
I20260809 03:35:24.286481 11057 leader_election.cc:302] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ae7ed0aa3984cb1bbba6a90fe4ff803; no voters:
I20260809 03:35:24.286883 11057 leader_election.cc:288] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:24.286963 11063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:24.287186 11063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [term 1 LEADER]: Becoming Leader. State: Replica: 1ae7ed0aa3984cb1bbba6a90fe4ff803, State: Running, Role: LEADER
I20260809 03:35:24.287513 11063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae7ed0aa3984cb1bbba6a90fe4ff803" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42077 } }
I20260809 03:35:24.288287 11057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:24.289426 11065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ae7ed0aa3984cb1bbba6a90fe4ff803" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae7ed0aa3984cb1bbba6a90fe4ff803" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42077 } } }
I20260809 03:35:24.289470 11064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ae7ed0aa3984cb1bbba6a90fe4ff803. Latest consensus state: current_term: 1 leader_uuid: "1ae7ed0aa3984cb1bbba6a90fe4ff803" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae7ed0aa3984cb1bbba6a90fe4ff803" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42077 } } }
I20260809 03:35:24.289565 11065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:24.289597 11064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:24.290094 11076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:24.294035 11076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:24.300549 11076 catalog_manager.cc:1347] Generated new cluster ID: 9c3d8856ae9043d792682af468091cad
I20260809 03:35:24.300655 11076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:24.307888 11076 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:24.309044 11076 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:24.313508 11076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ae7ed0aa3984cb1bbba6a90fe4ff803: Generated new TSK 0
I20260809 03:35:24.314323 11076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:35:24.328890 11060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.322908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58128 (local address 127.1.224.126:42077)
0809 03:35:24.323085 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.323090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.323280 (+ 190us) server_negotiation.cc:407] Connection header received
0809 03:35:24.323481 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.323484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.323542 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.323646 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0809 03:35:24.323953 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.324589 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.325933 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.326194 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.326284 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.326685 (+ 401us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 03:35:24.328413 (+ 1728us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36489/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 03:35:24.328719 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58128: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36489/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 03:35:24.328974 11060 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58128: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36489/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 03:35:24.330699 11103 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:24.322810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42077 (local address 127.0.0.1:58128)
0809 03:35:24.323176 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:24.323193 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:24.323322 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:24.323680 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:24.323684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:24.323693 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 03:35:24.323830 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.323837 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.324733 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.324738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.325779 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.325791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.326332 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.326337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.326431 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.326466 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 03:35:24.328624 (+ 2158us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36489/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 03:35:24.330504 (+ 1880us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.224.126:42077: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36489/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":1642,"thread_start_us":130,"threads_started":1}
I20260809 03:35:24.331789 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 10964
E20260809 03:35:24.347674 11104 webserver.cc:533] Webserver: sslize failed:
E20260809 03:35:24.347807 11104 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T03:35:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (335 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T03:35:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:24Z Disabled control of system clock
I20260809 03:35:24.397440 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:39777
--webserver_interface=127.1.224.126
--webserver_port=0
--builtin_ntp_servers=127.1.224.84:40947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:39777
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/testchainca.cert with env {}
W20260809 03:35:24.517139 11108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:24.517470 11108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:24.517521 11108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:24.521754 11108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:24.521838 11108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:24.521854 11108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:24.521871 11108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:24.521886 11108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 03:35:24.521903 11108 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 03:35:24.521915 11108 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 03:35:24.528025 11108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:40947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:39777
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:39777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:24.529381 11108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:24.530571 11108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:24.539988 11115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:24.540081 11119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:24.540163 11108 server_base.cc:1034] running on GCE node
W20260809 03:35:24.540024 11116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:24.540966 11108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:24.542258 11108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:24.543469 11108 hybrid_clock.cc:648] HybridClock initialized: now 1786246524543450 us; error 35 us; skew 500 ppm
I20260809 03:35:24.549829 11108 webserver.cc:460] Webserver started at http://127.1.224.126:40023/ using document root <none> and password file <none>
I20260809 03:35:24.550537 11108 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:24.550644 11108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:24.550908 11108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:24.553236 11108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "b8fcfc1ddd7a42a5ad33498969ebe578"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-jq4t"
server_key: "3c5f492fa52a789ce04334033fb34c28"
server_key_iv: "c7b4f69b802791916e87819fc2808af2"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.553848 11108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "b8fcfc1ddd7a42a5ad33498969ebe578"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-jq4t"
server_key: "3c5f492fa52a789ce04334033fb34c28"
server_key_iv: "c7b4f69b802791916e87819fc2808af2"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.557873 11108 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:35:24.560431 11129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:24.561792 11108 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:35:24.561954 11108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "b8fcfc1ddd7a42a5ad33498969ebe578"
format_stamp: "Formatted at 2026-08-09 03:35:24 on dist-test-slave-jq4t"
server_key: "3c5f492fa52a789ce04334033fb34c28"
server_key_iv: "c7b4f69b802791916e87819fc2808af2"
server_key_version: "encryptionkey@0"
I20260809 03:35:24.562093 11108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:24.580327 11108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:24.581207 11108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:24.581439 11108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:24.589480 11108 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:39777
I20260809 03:35:24.589486 11209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:39777 every 8 connection(s)
I20260809 03:35:24.590560 11108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:24.593784 11211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:24.594611 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 11108
I20260809 03:35:24.594730 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:24.595053 1921 external_mini_cluster.cc:1428] Setting key 167563058f0052b6ca691e2915996602
I20260809 03:35:24.600426 11211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578: Bootstrap starting.
I20260809 03:35:24.602855 11211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:24.603667 11211 log.cc:824] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:24.604964 11215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.596599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49524 (local address 127.1.224.126:39777)
0809 03:35:24.597302 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.597319 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.597357 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:35:24.598036 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.598054 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.598455 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.598812 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:24.599256 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.600252 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.601276 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.601779 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.601962 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.602899 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:24.602923 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:24.602945 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:24.603374 (+ 429us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:24.603431 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:24.603452 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:24.603578 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:24.603701 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 03:35:24.603979 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":144,"threads_started":1}
I20260809 03:35:24.606113 11211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578: No bootstrap required, opened a new log
I20260809 03:35:24.608059 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:35:24.610448 11211 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8fcfc1ddd7a42a5ad33498969ebe578" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39777 } }
I20260809 03:35:24.610737 11211 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:24.610849 11211 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8fcfc1ddd7a42a5ad33498969ebe578, State: Initialized, Role: FOLLOWER
I20260809 03:35:24.611340 11211 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8fcfc1ddd7a42a5ad33498969ebe578" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39777 } }
I20260809 03:35:24.611505 11211 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:24.611599 11211 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:24.611727 11211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:24.612916 11211 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8fcfc1ddd7a42a5ad33498969ebe578" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39777 } }
I20260809 03:35:24.613389 11211 leader_election.cc:302] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8fcfc1ddd7a42a5ad33498969ebe578; no voters:
I20260809 03:35:24.613845 11211 leader_election.cc:288] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:24.613938 11218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:24.614171 11218 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [term 1 LEADER]: Becoming Leader. State: Replica: b8fcfc1ddd7a42a5ad33498969ebe578, State: Running, Role: LEADER
I20260809 03:35:24.614507 11218 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8fcfc1ddd7a42a5ad33498969ebe578" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39777 } }
I20260809 03:35:24.615424 11211 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:24.616569 11219 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8fcfc1ddd7a42a5ad33498969ebe578. Latest consensus state: current_term: 1 leader_uuid: "b8fcfc1ddd7a42a5ad33498969ebe578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8fcfc1ddd7a42a5ad33498969ebe578" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39777 } } }
I20260809 03:35:24.616750 11219 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:24.617031 11220 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8fcfc1ddd7a42a5ad33498969ebe578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8fcfc1ddd7a42a5ad33498969ebe578" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39777 } } }
I20260809 03:35:24.617137 11220 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:24.617455 11227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:24.620651 11227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:24.626304 11227 catalog_manager.cc:1347] Generated new cluster ID: c128ce3d4d624df19c32836979084f86
I20260809 03:35:24.626391 11227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:24.641287 11227 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:24.642638 11227 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:24.652729 11227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8fcfc1ddd7a42a5ad33498969ebe578: Generated new TSK 0
I20260809 03:35:24.653581 11227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:35:24.682717 11215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:24.668570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49532 (local address 127.1.224.126:39777)
0809 03:35:24.668733 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:24.668738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:24.668947 (+ 209us) server_negotiation.cc:407] Connection header received
0809 03:35:24.669126 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:24.669129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:24.669185 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:24.669286 (+ 101us) server_negotiation.cc:226] Negotiated authn=JWT
0809 03:35:24.669581 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.670159 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.671468 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.671703 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.671778 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.672195 (+ 417us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 03:35:24.682168 (+ 9973us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 03:35:24.682557 (+ 389us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49532: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 03:35:24.682812 11215 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49532: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 03:35:24.682781 11250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:24.668440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39777 (local address 127.0.0.1:49532)
0809 03:35:24.668841 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:24.668857 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:24.668995 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:24.669320 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:24.669323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:24.669332 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 03:35:24.669460 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.669466 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.670279 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.670283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.671335 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:24.671346 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:24.671827 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:24.671831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:24.671927 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:24.671959 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 03:35:24.682430 (+ 10471us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 03:35:24.682620 (+ 190us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.224.126:39777: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":135,"threads_started":1}
I20260809 03:35:24.683861 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 11108
2026-08-09T03:35:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (325 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (139 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (134 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:25 dist-test-slave-jq4t krb5kdc[11271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:25 dist-test-slave-jq4t krb5kdc[11271](info): set up 2 sockets
Aug 09 03:35:25 dist-test-slave-jq4t krb5kdc[11271](info): commencing operation
krb5kdc: starting...
W20260809 03:35:26.223868 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:35:26 dist-test-slave-jq4t krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:35:26.363549 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:45253
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:41447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:45253
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:26.484467 11288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:26.484859 11288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:26.484935 11288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:26.489087 11288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:26.489203 11288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:26.489241 11288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:26.489277 11288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:26.489315 11288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:26.494973 11288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:41447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:45253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:45253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:26.496255 11288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:26.497442 11288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:26.506560 11295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:26.506560 11296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:26.506826 11298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:26.506872 11288 server_base.cc:1034] running on GCE node
I20260809 03:35:26.507611 11288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:26.508921 11288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:26.510097 11288 hybrid_clock.cc:648] HybridClock initialized: now 1786246526510077 us; error 37 us; skew 500 ppm
Aug 09 03:35:26 dist-test-slave-jq4t krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:26.513164 11288 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:35:26.514201 11288 webserver.cc:460] Webserver started at http://127.1.224.126:39437/ using document root <none> and password file <none>
I20260809 03:35:26.514797 11288 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:26.514863 11288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:26.515055 11288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:26.517402 11288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "ef8fa58cf14d4064a4ce6d000a80d4d8"
format_stamp: "Formatted at 2026-08-09 03:35:26 on dist-test-slave-jq4t"
server_key: "17bcc4c0d366f46e76cc676f2ce4e27f"
server_key_iv: "39bab609f9594fc2ee2164f7987849d9"
server_key_version: "encryptionkey@0"
I20260809 03:35:26.517962 11288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "ef8fa58cf14d4064a4ce6d000a80d4d8"
format_stamp: "Formatted at 2026-08-09 03:35:26 on dist-test-slave-jq4t"
server_key: "17bcc4c0d366f46e76cc676f2ce4e27f"
server_key_iv: "39bab609f9594fc2ee2164f7987849d9"
server_key_version: "encryptionkey@0"
I20260809 03:35:26.522064 11288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:35:26.524578 11311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:26.525957 11288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:26.526080 11288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "ef8fa58cf14d4064a4ce6d000a80d4d8"
format_stamp: "Formatted at 2026-08-09 03:35:26 on dist-test-slave-jq4t"
server_key: "17bcc4c0d366f46e76cc676f2ce4e27f"
server_key_iv: "39bab609f9594fc2ee2164f7987849d9"
server_key_version: "encryptionkey@0"
I20260809 03:35:26.526177 11288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:26.538578 11288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:26.541873 11288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:26.542050 11288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:26.549731 11288 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:45253
I20260809 03:35:26.549742 11390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:45253 every 8 connection(s)
I20260809 03:35:26.550786 11288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:26.551826 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 11288
I20260809 03:35:26.551975 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:26.552312 1921 external_mini_cluster.cc:1428] Setting key 3d96eeeaf94cde445ce64d4506cec855
I20260809 03:35:26.554657 11391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:26.562228 11391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8: Bootstrap starting.
Aug 09 03:35:26 dist-test-slave-jq4t krb5kdc[11271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:26.564802 11391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:26.565583 11391 log.cc:824] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:26.568002 11391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8: No bootstrap required, opened a new log
I20260809 03:35:26.568749 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:26.554047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50464 (local address 127.1.224.126:45253)
0809 03:35:26.554574 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:26.554590 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:26.554630 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:35:26.555416 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:26.555437 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:26.555792 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:26.556097 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:26.557158 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:26.558251 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:26.559464 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:26.559988 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:26.560130 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:26.563389 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:26.563395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:26.563411 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:26.563442 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:26.565393 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:26.565908 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:26.565913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:26.565918 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:26.566087 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:26.566413 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:26.566420 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:26.566424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:26.566891 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:26.567042 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:26.567355 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 03:35:26.567665 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":180,"threads_started":1}
I20260809 03:35:26.571142 11391 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8fa58cf14d4064a4ce6d000a80d4d8" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45253 } }
I20260809 03:35:26.571398 11391 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:26.571494 11391 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8fa58cf14d4064a4ce6d000a80d4d8, State: Initialized, Role: FOLLOWER
I20260809 03:35:26.571933 11391 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8fa58cf14d4064a4ce6d000a80d4d8" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45253 } }
I20260809 03:35:26.572088 11391 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:26.572176 11391 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:26.572299 11391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:26.573311 11391 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8fa58cf14d4064a4ce6d000a80d4d8" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45253 } }
I20260809 03:35:26.573697 11391 leader_election.cc:302] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef8fa58cf14d4064a4ce6d000a80d4d8; no voters:
I20260809 03:35:26.574087 11391 leader_election.cc:288] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:26.574169 11396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:26.574368 11396 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [term 1 LEADER]: Becoming Leader. State: Replica: ef8fa58cf14d4064a4ce6d000a80d4d8, State: Running, Role: LEADER
I20260809 03:35:26.574703 11396 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8fa58cf14d4064a4ce6d000a80d4d8" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45253 } }
I20260809 03:35:26.575450 11391 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:26.576582 11399 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef8fa58cf14d4064a4ce6d000a80d4d8. Latest consensus state: current_term: 1 leader_uuid: "ef8fa58cf14d4064a4ce6d000a80d4d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8fa58cf14d4064a4ce6d000a80d4d8" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45253 } } }
I20260809 03:35:26.576730 11399 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:26.577087 11398 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef8fa58cf14d4064a4ce6d000a80d4d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8fa58cf14d4064a4ce6d000a80d4d8" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45253 } } }
I20260809 03:35:26.577245 11398 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:26.577147 11410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:26.580688 11410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:26.586874 11410 catalog_manager.cc:1347] Generated new cluster ID: 60ae55b1a85749a9a1ed61b6c646888d
I20260809 03:35:26.586963 11410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:26.596521 11410 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:26.597909 11410 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:26.606549 11410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef8fa58cf14d4064a4ce6d000a80d4d8: Generated new TSK 0
I20260809 03:35:26.607312 11410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:35:26.644858 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45253
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:41447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:26.766773 11428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:26.767089 11428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:26.767135 11428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:26.771215 11428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:26.771286 11428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:26.771353 11428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:26.777060 11428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:41447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:26.778195 11428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:26.779256 11428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:26.789245 11433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:26.789245 11436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:26.789269 11439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:26.789636 11428 server_base.cc:1034] running on GCE node
I20260809 03:35:26.790122 11428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:26.790781 11428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:26.791993 11428 hybrid_clock.cc:648] HybridClock initialized: now 1786246526791975 us; error 38 us; skew 500 ppm
Aug 09 03:35:26 dist-test-slave-jq4t krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:26.795029 11428 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:35:26.795989 11428 webserver.cc:460] Webserver started at http://127.1.224.65:45673/ using document root <none> and password file <none>
I20260809 03:35:26.796682 11428 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:26.796779 11428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:26.797030 11428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:26.799248 11428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "401ff7fa5b5b4e0b8d57b2f862f3fef2"
format_stamp: "Formatted at 2026-08-09 03:35:26 on dist-test-slave-jq4t"
server_key: "fe9e9cdcc03d96dd352df98277ca9d4a"
server_key_iv: "5db043c0c03706a03b4ee7a3e4b89db0"
server_key_version: "encryptionkey@0"
I20260809 03:35:26.799860 11428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "401ff7fa5b5b4e0b8d57b2f862f3fef2"
format_stamp: "Formatted at 2026-08-09 03:35:26 on dist-test-slave-jq4t"
server_key: "fe9e9cdcc03d96dd352df98277ca9d4a"
server_key_iv: "5db043c0c03706a03b4ee7a3e4b89db0"
server_key_version: "encryptionkey@0"
I20260809 03:35:26.804029 11428 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 03:35:26.807395 11449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:26.808703 11428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:26.808846 11428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "401ff7fa5b5b4e0b8d57b2f862f3fef2"
format_stamp: "Formatted at 2026-08-09 03:35:26 on dist-test-slave-jq4t"
server_key: "fe9e9cdcc03d96dd352df98277ca9d4a"
server_key_iv: "5db043c0c03706a03b4ee7a3e4b89db0"
server_key_version: "encryptionkey@0"
I20260809 03:35:26.808969 11428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:26.818655 11428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:26.822139 11428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:26.822363 11428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:26.823083 11428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:26.823982 11428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:26.824054 11428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:26.824129 11428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:26.824182 11428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:26.834860 11428 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:41167
I20260809 03:35:26.834882 11613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:41167 every 8 connection(s)
I20260809 03:35:26.835924 11428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:26.843397 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 11428
I20260809 03:35:26.843550 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:26.843866 1921 external_mini_cluster.cc:1428] Setting key d4b4b6f6ea17bcf71f07d3a85de0b760
Aug 09 03:35:26 dist-test-slave-jq4t krb5kdc[11271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246526, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:26.851182 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:26.837996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:43599 (local address 127.1.224.126:45253)
0809 03:35:26.838160 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:26.838164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:26.839063 (+ 899us) server_negotiation.cc:407] Connection header received
0809 03:35:26.840685 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:26.840689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:26.840756 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:26.840861 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:26.842226 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:26.842798 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:26.844172 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:26.844556 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:26.844673 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:26.847338 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:26.847342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:26.847345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:26.847377 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:26.848939 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:26.849475 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:26.849479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:26.849481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:26.849566 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:26.849961 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:26.849964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:26.849966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:26.850191 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:26.850301 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:26.850850 (+ 549us) server_negotiation.cc:299] Negotiation successful
0809 03:35:26.851001 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 03:35:26.852053 11618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:26.838397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:45253 (local address 127.1.224.65:43599)
0809 03:35:26.838897 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:26.838942 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:26.840396 (+ 1454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:26.841060 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:26.841067 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:26.841439 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:26.841994 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:26.842018 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:26.842935 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:26.842939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:26.844044 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:26.844054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:26.844732 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:26.844736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:26.844927 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:26.845602 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:26.845626 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:26.847171 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:26.849072 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:26.849079 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:26.849095 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:26.849366 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:26.849662 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:26.849666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:26.849668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:26.849828 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:26.850304 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:26.850314 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:26.850594 (+ 280us) client_negotiation.cc:769] Sending connection context
0809 03:35:26.850795 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 03:35:26.851166 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":113,"threads_started":1}
I20260809 03:35:26.853560 11616 heartbeater.cc:347] Connected to a master server at 127.1.224.126:45253
I20260809 03:35:26.853825 11616 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:26.854365 11616 heartbeater.cc:511] Master 127.1.224.126:45253 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:35:26.856048 11339 ts_manager.cc:194] Registered new tserver with Master: 401ff7fa5b5b4e0b8d57b2f862f3fef2 (127.1.224.65:41167)
I20260809 03:35:26.857419 11339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:43599
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:35:26.876137 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45253
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:41447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:26.996824 11624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:26.997105 11624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:26.997149 11624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:27.001286 11624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:27.001354 11624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:27.001413 11624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:27.006985 11624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:41447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:27.008239 11624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:27.009402 11624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:27.019518 11633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:27.019591 11631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:27.019618 11630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:27.019924 11624 server_base.cc:1034] running on GCE node
I20260809 03:35:27.020424 11624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:27.021123 11624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:27.022320 11624 hybrid_clock.cc:648] HybridClock initialized: now 1786246527022282 us; error 54 us; skew 500 ppm
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:27.025420 11624 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:35:27.026401 11624 webserver.cc:460] Webserver started at http://127.1.224.66:40745/ using document root <none> and password file <none>
I20260809 03:35:27.027269 11624 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:27.027372 11624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:27.027622 11624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:27.029932 11624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "59d324cc80ec4a7f8b7d6841cb9b9707"
format_stamp: "Formatted at 2026-08-09 03:35:27 on dist-test-slave-jq4t"
server_key: "70c589109dbe1553e5038b8cd40e3575"
server_key_iv: "0ee29aa14eb3c8a15e2a2c044824cc1f"
server_key_version: "encryptionkey@0"
I20260809 03:35:27.030591 11624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "59d324cc80ec4a7f8b7d6841cb9b9707"
format_stamp: "Formatted at 2026-08-09 03:35:27 on dist-test-slave-jq4t"
server_key: "70c589109dbe1553e5038b8cd40e3575"
server_key_iv: "0ee29aa14eb3c8a15e2a2c044824cc1f"
server_key_version: "encryptionkey@0"
I20260809 03:35:27.034601 11624 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 03:35:27.037253 11648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:27.038553 11624 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:35:27.038684 11624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "59d324cc80ec4a7f8b7d6841cb9b9707"
format_stamp: "Formatted at 2026-08-09 03:35:27 on dist-test-slave-jq4t"
server_key: "70c589109dbe1553e5038b8cd40e3575"
server_key_iv: "0ee29aa14eb3c8a15e2a2c044824cc1f"
server_key_version: "encryptionkey@0"
I20260809 03:35:27.038805 11624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:27.060005 11624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:27.063694 11624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:27.063913 11624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:27.064611 11624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:27.065508 11624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:27.065558 11624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:27.065629 11624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:27.065667 11624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:27.074704 11624 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:39299
I20260809 03:35:27.074733 11824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:39299 every 8 connection(s)
I20260809 03:35:27.075748 11624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:27.084697 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 11624
I20260809 03:35:27.084832 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:27.085173 1921 external_mini_cluster.cc:1428] Setting key 5aefa33ab7943f79cf29a1a6fe241f5f
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:27.092185 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:27.077991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:41015 (local address 127.1.224.126:45253)
0809 03:35:27.078135 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:27.078140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:27.079150 (+ 1010us) server_negotiation.cc:407] Connection header received
0809 03:35:27.080667 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:27.080671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:27.080735 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:27.080840 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:27.082098 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.082779 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.084072 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.084311 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.084427 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:27.087682 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:27.087685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:27.087687 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:27.087711 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:27.089619 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:27.090227 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:27.090233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:27.090236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:27.090349 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:27.090712 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:27.090718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:27.090722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:27.091034 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:27.091148 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:27.091743 (+ 595us) server_negotiation.cc:299] Negotiation successful
0809 03:35:27.091899 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:35:27.093016 11832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:27.078495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:45253 (local address 127.1.224.66:41015)
0809 03:35:27.078985 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:27.079030 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:27.080387 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:27.081037 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:27.081045 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:27.081430 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:27.081945 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:27.081958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.082922 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.082926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:27.083938 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:27.083948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.084512 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.084517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:27.084705 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:27.085401 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:27.085418 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:27.087460 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:27.089763 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:27.089769 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:27.089785 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:27.090104 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:27.090455 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:27.090458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:27.090459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:27.090611 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:27.091172 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:27.091179 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:27.091467 (+ 288us) client_negotiation.cc:769] Sending connection context
0809 03:35:27.091652 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 03:35:27.091969 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":122,"threads_started":1}
I20260809 03:35:27.094388 11826 heartbeater.cc:347] Connected to a master server at 127.1.224.126:45253
I20260809 03:35:27.094694 11826 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:27.095378 11826 heartbeater.cc:511] Master 127.1.224.126:45253 requested a full tablet report, sending...
I20260809 03:35:27.096668 11339 ts_manager.cc:194] Registered new tserver with Master: 59d324cc80ec4a7f8b7d6841cb9b9707 (127.1.224.66:39299)
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:35:27.097383 11339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:41015
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:35:27.115886 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45253
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:41447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:27.236820 11836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:27.237147 11836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:27.237221 11836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:27.241391 11836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:27.241477 11836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:27.241564 11836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:27.247218 11836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:41447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:27.248384 11836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:27.249516 11836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:27.259642 11841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:27.259646 11843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:27.259788 11836 server_base.cc:1034] running on GCE node
W20260809 03:35:27.259657 11847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:27.260409 11836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:27.261137 11836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:27.262332 11836 hybrid_clock.cc:648] HybridClock initialized: now 1786246527262313 us; error 28 us; skew 500 ppm
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:27.265376 11836 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:35:27.266392 11836 webserver.cc:460] Webserver started at http://127.1.224.67:38729/ using document root <none> and password file <none>
I20260809 03:35:27.267030 11836 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:27.267128 11836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:27.267387 11836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:27.269717 11836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "71ece53423934b09934d4e954b85bf94"
format_stamp: "Formatted at 2026-08-09 03:35:27 on dist-test-slave-jq4t"
server_key: "447a6d250aa4fe822ed9983752e112a2"
server_key_iv: "156b60185c8a703397e835eb3810ca44"
server_key_version: "encryptionkey@0"
I20260809 03:35:27.270337 11836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "71ece53423934b09934d4e954b85bf94"
format_stamp: "Formatted at 2026-08-09 03:35:27 on dist-test-slave-jq4t"
server_key: "447a6d250aa4fe822ed9983752e112a2"
server_key_iv: "156b60185c8a703397e835eb3810ca44"
server_key_version: "encryptionkey@0"
I20260809 03:35:27.274529 11836 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:27.277717 11859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:27.279093 11836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:35:27.279372 11836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "71ece53423934b09934d4e954b85bf94"
format_stamp: "Formatted at 2026-08-09 03:35:27 on dist-test-slave-jq4t"
server_key: "447a6d250aa4fe822ed9983752e112a2"
server_key_iv: "156b60185c8a703397e835eb3810ca44"
server_key_version: "encryptionkey@0"
I20260809 03:35:27.279517 11836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:27.299908 11836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:27.303658 11836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:27.303884 11836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:27.304626 11836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:27.305522 11836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:27.305593 11836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:27.305667 11836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:27.305720 11836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:27.315623 11836 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:46807
I20260809 03:35:27.315646 12038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:46807 every 8 connection(s)
I20260809 03:35:27.316668 11836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:27.324041 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 11836
I20260809 03:35:27.324178 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:27.324532 1921 external_mini_cluster.cc:1428] Setting key 6e50470f208ed4a804f3b21d78cb3888
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:27.332441 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:27.318612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:35879 (local address 127.1.224.126:45253)
0809 03:35:27.318765 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:27.318768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:27.319623 (+ 855us) server_negotiation.cc:407] Connection header received
0809 03:35:27.321116 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:27.321120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:27.321183 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:27.321288 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:27.322499 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.323065 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.324626 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.324879 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.325015 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:27.328265 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:27.328269 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:27.328271 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:27.328300 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:27.330045 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:27.330616 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:27.330619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:27.330620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:27.330710 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:27.331223 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:27.331226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:27.331227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:27.331449 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:27.331537 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:27.332152 (+ 615us) server_negotiation.cc:299] Negotiation successful
0809 03:35:27.332295 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:35:27.333431 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:27.319008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:45253 (local address 127.1.224.67:35879)
0809 03:35:27.319464 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:27.319507 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:27.320884 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:27.321466 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:27.321473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:27.321845 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:27.322344 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:27.322358 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.323211 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.323214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:27.324445 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:27.324459 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.325074 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.325079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:27.325298 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:27.326038 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:27.326055 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:27.328058 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:27.330174 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:27.330181 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:27.330199 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:27.330506 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:27.330824 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:27.330846 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:27.330848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:27.331127 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:27.331552 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:27.331558 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:27.331864 (+ 306us) client_negotiation.cc:769] Sending connection context
0809 03:35:27.332065 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 03:35:27.332386 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":108,"threads_started":1}
I20260809 03:35:27.334858 12039 heartbeater.cc:347] Connected to a master server at 127.1.224.126:45253
I20260809 03:35:27.335186 12039 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:27.335803 12039 heartbeater.cc:511] Master 127.1.224.126:45253 requested a full tablet report, sending...
I20260809 03:35:27.337316 11339 ts_manager.cc:194] Registered new tserver with Master: 71ece53423934b09934d4e954b85bf94 (127.1.224.67:46807)
I20260809 03:35:27.338091 11339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:35879
I20260809 03:35:27.339236 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/test-admin.keytab.
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:27.357915 1921 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 03:35:27.360993 12048 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:27.359461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35870 (local address 127.1.224.65:41167)
0809 03:35:27.359805 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:27.359818 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:27.359846 (+ 28us) server_negotiation.cc:407] Connection header received
0809 03:35:27.359901 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:27.359908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:27.360115 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:27.360243 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:27.360791 (+ 548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35870: BlockingRecv error: recv got EOF from 127.0.0.1:35870 (error 108)
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":121,"threads_started":1}
W20260809 03:35:27.361142 1921 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/krb5cc' of type FILE
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:27.362287 1921 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:27.363268 1921 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.65@KRBTEST.COM
I20260809 03:35:27.372994 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:27.364260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.1.224.65:41167)
0809 03:35:27.364409 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:27.364413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:27.364450 (+ 37us) server_negotiation.cc:407] Connection header received
0809 03:35:27.364630 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:27.364633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:27.364677 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:27.364746 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:27.365272 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.366038 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.366841 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.367221 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.367302 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:27.369590 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:27.369600 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:27.369615 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:27.369640 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:27.371216 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:27.371636 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:27.371641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:27.371643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:27.371723 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:27.371956 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:27.371958 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:27.371959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:27.372213 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:27.372382 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:27.372632 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 03:35:27.372805 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 03:35:27.377424 12048 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:27.376118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35890 (local address 127.1.224.65:41167)
0809 03:35:27.376234 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:27.376239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:27.376396 (+ 157us) server_negotiation.cc:407] Connection header received
0809 03:35:27.376561 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:27.376564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:27.376617 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:27.376682 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:27.377290 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35890: BlockingRecv error: recv got EOF from 127.0.0.1:35890 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260809 03:35:27.377485 1921 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/krb5cc' of type FILE
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:27.378589 1921 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:27.379663 1921 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.65@KRBTEST.COM
I20260809 03:35:27.392001 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:27.380866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35898 (local address 127.1.224.65:41167)
0809 03:35:27.381043 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:27.381048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:27.383665 (+ 2617us) server_negotiation.cc:407] Connection header received
0809 03:35:27.383726 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:27.383732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:27.383783 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:27.383865 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:27.384554 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.385348 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.386275 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.386586 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.386708 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:27.388993 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:27.388997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:27.389000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:27.389031 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:27.390247 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:27.390674 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:27.390679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:27.390681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:27.390773 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:27.391098 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:27.391102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:27.391103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:27.391296 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:27.391465 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:27.391697 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 03:35:27.391822 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 03:35:27.395202 12048 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:27.393891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35912 (local address 127.1.224.65:41167)
0809 03:35:27.394014 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:27.394018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:27.394175 (+ 157us) server_negotiation.cc:407] Connection header received
0809 03:35:27.394258 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:27.394260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:27.394296 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:27.394360 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:27.395074 (+ 714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35912: BlockingRecv error: recv got EOF from 127.0.0.1:35912 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 03:35:27.395303 1921 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786246494448707-1921-0/krb5kdc/krb5cc' of type FILE
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:27.396400 1921 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:27.397411 1921 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[11271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.65@KRBTEST.COM
I20260809 03:35:27.407716 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:27.398478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35914 (local address 127.1.224.65:41167)
0809 03:35:27.398647 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:27.398651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:27.398666 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:35:27.398837 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:27.398840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:27.398881 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:27.398972 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:27.399604 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.400114 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.401035 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:27.401270 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:27.401353 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:27.403916 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:27.403920 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:27.403923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:27.403961 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:27.405708 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:27.406209 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:27.406212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:27.406213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:27.406300 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:27.406657 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:27.406661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:27.406662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:27.406873 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:27.407108 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:27.407352 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 03:35:27.407466 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 03:35:27.409035 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 11428
I20260809 03:35:27.425203 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 11624
I20260809 03:35:27.447754 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 11836
I20260809 03:35:27.472927 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 11288
2026-08-09T03:35:27Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2517 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[12065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[12065](info): set up 2 sockets
Aug 09 03:35:27 dist-test-slave-jq4t krb5kdc[12065](info): commencing operation
krb5kdc: starting...
W20260809 03:35:28.858290 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.353s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:35:28 dist-test-slave-jq4t krb5kdc[12065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:28Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.126.keytab.
Entry for principal oryx/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.126.keytab.
I20260809 03:35:28.992278 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:39163
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.126.keytab
--principal=oryx/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:39163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:29.113076 12086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:29.113389 12086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:29.113473 12086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:29.117537 12086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:29.117623 12086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:29.117661 12086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:29.117722 12086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:29.117760 12086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:29.123174 12086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:39163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.126.keytab
--principal=oryx/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:39163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:29.124346 12086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:29.125463 12086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:29.134498 12092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:29.134505 12095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:29.134611 12086 server_base.cc:1034] running on GCE node
W20260809 03:35:29.134514 12093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:29.135455 12086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:29.136691 12086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:29.137861 12086 hybrid_clock.cc:648] HybridClock initialized: now 1786246529137838 us; error 37 us; skew 500 ppm
Aug 09 03:35:29 dist-test-slave-jq4t krb5kdc[12065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246529, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:29.140769 12086 init.cc:377] Logged in from keytab as oryx/127.1.224.126@KRBTEST.COM (short username oryx)
I20260809 03:35:29.141768 12086 webserver.cc:460] Webserver started at http://127.1.224.126:39575/ using document root <none> and password file <none>
I20260809 03:35:29.142370 12086 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:29.142464 12086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:29.142714 12086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:29.144997 12086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "418b6f7249104662b195da37a5772666"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "b2aa28a8a8bfff2b467a12bb01d9d6fc"
server_key_iv: "05d130adf378fde3e75005888c129c45"
server_key_version: "encryptionkey@0"
I20260809 03:35:29.145586 12086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "418b6f7249104662b195da37a5772666"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "b2aa28a8a8bfff2b467a12bb01d9d6fc"
server_key_iv: "05d130adf378fde3e75005888c129c45"
server_key_version: "encryptionkey@0"
I20260809 03:35:29.149636 12086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:35:29.152148 12104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:29.153492 12086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:29.153659 12086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "418b6f7249104662b195da37a5772666"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "b2aa28a8a8bfff2b467a12bb01d9d6fc"
server_key_iv: "05d130adf378fde3e75005888c129c45"
server_key_version: "encryptionkey@0"
I20260809 03:35:29.153789 12086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:29.171244 12086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:29.174636 12086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:29.174883 12086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:29.182904 12086 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:39163
I20260809 03:35:29.183307 12184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:39163 every 8 connection(s)
I20260809 03:35:29.184044 12086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:29.187347 12186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:29.189754 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 12086
I20260809 03:35:29.189862 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:29.190274 1921 external_mini_cluster.cc:1428] Setting key 988002828295d5016c5038912bf3fcd6
I20260809 03:35:29.194113 12186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666: Bootstrap starting.
I20260809 03:35:29.196547 12186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:29.197522 12186 log.cc:824] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:35:29 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:29.199887 12186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666: No bootstrap required, opened a new log
I20260809 03:35:29.204694 12186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } }
I20260809 03:35:29.204964 12186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:29.205037 12186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418b6f7249104662b195da37a5772666, State: Initialized, Role: FOLLOWER
I20260809 03:35:29.205530 12186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } }
I20260809 03:35:29.205669 12186 raft_consensus.cc:388] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:29.205763 12186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:29.205869 12186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:29.206033 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:29.191774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.1.224.126:39163)
0809 03:35:29.192275 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:29.192291 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:29.192329 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:35:29.193163 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:29.193187 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:29.193538 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:29.193883 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:29.194746 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.195780 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.196739 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.197470 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.197672 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:29.200583 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:29.200588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:29.200604 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:29.200632 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:29.202462 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:29.203033 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:29.203038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:29.203043 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:29.203163 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:29.203464 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:29.203467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:29.203468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:29.203912 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:29.204084 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:29.204535 (+ 451us) server_negotiation.cc:299] Negotiation successful
0809 03:35:29.204893 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":150,"threads_started":1}
I20260809 03:35:29.206969 12186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } }
I20260809 03:35:29.207394 12186 leader_election.cc:302] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 418b6f7249104662b195da37a5772666; no voters:
I20260809 03:35:29.207758 12186 leader_election.cc:288] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:29.207878 12195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:29.208110 12195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 1 LEADER]: Becoming Leader. State: Replica: 418b6f7249104662b195da37a5772666, State: Running, Role: LEADER
I20260809 03:35:29.208866 12195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } }
I20260809 03:35:29.209208 12186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:29.210500 12196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } }
I20260809 03:35:29.210657 12196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:29.211046 12197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 418b6f7249104662b195da37a5772666. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } }
I20260809 03:35:29.211131 12197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:29.211139 12210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:29.214460 12210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:29.220345 12210 catalog_manager.cc:1347] Generated new cluster ID: 83dc0a9e3c9e46ad9d196726bcc3072b
I20260809 03:35:29.220427 12210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:29.233225 12210 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:29.234241 12210 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:29.242830 12210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666: Generated new TSK 0
I20260809 03:35:29.243665 12210 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab.
Entry for principal oryx/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab.
I20260809 03:35:29.280659 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39163
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab
--principal=oryx/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:29.406965 12222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:29.407260 12222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:29.407336 12222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:29.411836 12222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:29.411922 12222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:29.411986 12222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:29.417717 12222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab
--principal=oryx/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:29.419024 12222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:29.420099 12222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:29.430230 12232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:29.430218 12229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:29.430451 12234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:29.430497 12222 server_base.cc:1034] running on GCE node
I20260809 03:35:29.430943 12222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:29.431658 12222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:29.432823 12222 hybrid_clock.cc:648] HybridClock initialized: now 1786246529432810 us; error 26 us; skew 500 ppm
Aug 09 03:35:29 dist-test-slave-jq4t krb5kdc[12065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246529, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:29.435796 12222 init.cc:377] Logged in from keytab as oryx/127.1.224.65@KRBTEST.COM (short username oryx)
I20260809 03:35:29.436826 12222 webserver.cc:460] Webserver started at http://127.1.224.65:39417/ using document root <none> and password file <none>
I20260809 03:35:29.437399 12222 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:29.437466 12222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:29.437646 12222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:29.439872 12222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "bca54a7b094841a5883552c27c0e94c0"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "8e445c460699c9a3adb727c44405e4d6"
server_key_iv: "dae532e132148867a04312540f886e02"
server_key_version: "encryptionkey@0"
I20260809 03:35:29.440413 12222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "bca54a7b094841a5883552c27c0e94c0"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "8e445c460699c9a3adb727c44405e4d6"
server_key_iv: "dae532e132148867a04312540f886e02"
server_key_version: "encryptionkey@0"
I20260809 03:35:29.444555 12222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:35:29.447134 12243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:29.448596 12222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:29.448758 12222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "bca54a7b094841a5883552c27c0e94c0"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "8e445c460699c9a3adb727c44405e4d6"
server_key_iv: "dae532e132148867a04312540f886e02"
server_key_version: "encryptionkey@0"
I20260809 03:35:29.448899 12222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:29.470072 12222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:29.473882 12222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:29.474134 12222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:29.474825 12222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:29.475746 12222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:29.475823 12222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:29.475910 12222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:29.475963 12222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:29.485700 12222 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:33961
I20260809 03:35:29.485723 12413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:33961 every 8 connection(s)
I20260809 03:35:29.486685 12222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:29.489377 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 12222
I20260809 03:35:29.489508 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:29.489849 1921 external_mini_cluster.cc:1428] Setting key a46e766c2cb3e389879d0dee6e2fcefc
Aug 09 03:35:29 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246529, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:35:29.503589 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:29.488757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:34473 (local address 127.1.224.126:39163)
0809 03:35:29.488907 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:29.488911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:29.489955 (+ 1044us) server_negotiation.cc:407] Connection header received
0809 03:35:29.491429 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:29.491433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:29.491491 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:29.491575 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:29.493333 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.493942 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.495267 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.495510 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.495612 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:29.499238 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:29.499245 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:29.499251 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:29.499290 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:29.501152 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:29.501741 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:29.501748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:29.501751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:29.501856 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:29.502350 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:29.502356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:29.502359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:29.502637 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:29.502725 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:29.503319 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 03:35:29.503447 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:35:29.504729 12416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:29.489208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.65:34473)
0809 03:35:29.489780 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:29.489842 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:29.491218 (+ 1376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:29.491796 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:29.491805 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:29.492339 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:29.493144 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:29.493167 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.494096 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.494104 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:29.495102 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:29.495119 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.495699 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.495706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:29.495966 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:29.496777 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:29.496807 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:29.499013 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:29.501283 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:29.501289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:29.501305 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:29.501602 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:29.501983 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:29.501986 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:29.501988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:29.502168 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:29.502745 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:29.502751 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:29.503050 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 03:35:29.503225 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 03:35:29.503673 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":150,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.66@KRBTEST.COM" created.
I20260809 03:35:29.506421 12414 heartbeater.cc:347] Connected to a master server at 127.1.224.126:39163
I20260809 03:35:29.506742 12414 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:29.507385 12414 heartbeater.cc:511] Master 127.1.224.126:39163 requested a full tablet report, sending...
I20260809 03:35:29.509380 12130 ts_manager.cc:194] Registered new tserver with Master: bca54a7b094841a5883552c27c0e94c0 (127.1.224.65:33961)
I20260809 03:35:29.510764 12130 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:34473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab.
Entry for principal oryx/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab.
I20260809 03:35:29.525092 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39163
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab
--principal=oryx/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:29.644471 12421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:29.644868 12421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:29.644943 12421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:29.649174 12421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:29.649264 12421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:29.649353 12421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:29.654892 12421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab
--principal=oryx/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:29.656108 12421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:29.657231 12421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:29.667428 12428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:29.667441 12431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:29.667423 12429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:29.667766 12421 server_base.cc:1034] running on GCE node
I20260809 03:35:29.668211 12421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:29.668944 12421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:29.670111 12421 hybrid_clock.cc:648] HybridClock initialized: now 1786246529670097 us; error 27 us; skew 500 ppm
Aug 09 03:35:29 dist-test-slave-jq4t krb5kdc[12065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246529, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:29.675559 12421 init.cc:377] Logged in from keytab as oryx/127.1.224.66@KRBTEST.COM (short username oryx)
I20260809 03:35:29.676466 12421 webserver.cc:460] Webserver started at http://127.1.224.66:33851/ using document root <none> and password file <none>
I20260809 03:35:29.677129 12421 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:29.677194 12421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:29.677431 12421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:29.679610 12421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "eab5b9e49845493fb3bab869bc385008"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "69acf224b60cd25153e76765c12be8f9"
server_key_iv: "2754c2e9ed3f7e68a56c85bf4e842a48"
server_key_version: "encryptionkey@0"
I20260809 03:35:29.680204 12421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "eab5b9e49845493fb3bab869bc385008"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "69acf224b60cd25153e76765c12be8f9"
server_key_iv: "2754c2e9ed3f7e68a56c85bf4e842a48"
server_key_version: "encryptionkey@0"
I20260809 03:35:29.684266 12421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 03:35:29.686830 12443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:29.688062 12421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:29.688206 12421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "eab5b9e49845493fb3bab869bc385008"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "69acf224b60cd25153e76765c12be8f9"
server_key_iv: "2754c2e9ed3f7e68a56c85bf4e842a48"
server_key_version: "encryptionkey@0"
I20260809 03:35:29.688319 12421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:29.702541 12421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:29.706184 12421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:29.706398 12421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:29.707042 12421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:29.707931 12421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:29.707983 12421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:29.708055 12421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:29.708097 12421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:29.717905 12421 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:46675
I20260809 03:35:29.717926 12607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:46675 every 8 connection(s)
I20260809 03:35:29.718914 12421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:29.723302 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 12421
I20260809 03:35:29.723445 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:29.723794 1921 external_mini_cluster.cc:1428] Setting key 4386d80e9c26f87b79cd4d4feb01c2d3
Aug 09 03:35:29 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246529, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:29.735221 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:29.720962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:43103 (local address 127.1.224.126:39163)
0809 03:35:29.721122 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:29.721127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:29.721998 (+ 871us) server_negotiation.cc:407] Connection header received
0809 03:35:29.723391 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:29.723395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:29.723461 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:29.723575 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:29.725144 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.725775 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.727603 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.727995 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.728121 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:29.730862 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:29.730867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:29.730871 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:29.730907 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:29.732882 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:29.733422 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:29.733428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:29.733432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:29.733555 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:29.733908 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:29.733912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:29.733914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:29.734200 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:29.734320 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:29.734861 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 03:35:29.735022 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for oryx/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:35:29.736052 12614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:29.721316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.66:43103)
0809 03:35:29.721808 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:29.721856 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:29.723144 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:29.723815 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:29.723826 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:29.724340 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:29.724986 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:29.725015 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.725919 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.725925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:29.727448 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:29.727463 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.728158 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.728161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:29.728314 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:29.729022 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:29.729041 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:29.730634 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:29.733010 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:29.733015 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:29.733031 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:29.733309 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:29.733663 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:29.733665 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:29.733666 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:29.733807 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:29.734305 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:29.734311 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:29.734584 (+ 273us) client_negotiation.cc:769] Sending connection context
0809 03:35:29.734752 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 03:35:29.735057 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":124,"threads_started":1}
I20260809 03:35:29.737630 12608 heartbeater.cc:347] Connected to a master server at 127.1.224.126:39163
I20260809 03:35:29.737892 12608 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.67@KRBTEST.COM" created.
I20260809 03:35:29.738533 12608 heartbeater.cc:511] Master 127.1.224.126:39163 requested a full tablet report, sending...
I20260809 03:35:29.739739 12130 ts_manager.cc:194] Registered new tserver with Master: eab5b9e49845493fb3bab869bc385008 (127.1.224.66:46675)
I20260809 03:35:29.740361 12130 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:43103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab.
Entry for principal oryx/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab.
I20260809 03:35:29.756664 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39163
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab
--principal=oryx/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:29.876994 12620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:29.877287 12620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:29.877329 12620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:29.881438 12620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:29.881513 12620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:29.881572 12620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:29.887017 12620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab
--principal=oryx/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:29.888149 12620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:29.889207 12620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:29.899618 12633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:29.899618 12630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:29.899812 12620 server_base.cc:1034] running on GCE node
W20260809 03:35:29.899639 12631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:29.900410 12620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:29.901146 12620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:29.902349 12620 hybrid_clock.cc:648] HybridClock initialized: now 1786246529902330 us; error 35 us; skew 500 ppm
Aug 09 03:35:29 dist-test-slave-jq4t krb5kdc[12065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246529, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:29.905611 12620 init.cc:377] Logged in from keytab as oryx/127.1.224.67@KRBTEST.COM (short username oryx)
I20260809 03:35:29.906718 12620 webserver.cc:460] Webserver started at http://127.1.224.67:46135/ using document root <none> and password file <none>
I20260809 03:35:29.907354 12620 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:29.907454 12620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:29.907718 12620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:29.910151 12620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "a4f7a00046b74bffb36c8796ffa67be1"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "0a8cea7f5f53194206f590f8e016b414"
server_key_iv: "ffb7063cd017d7004e5dadbb9bfeed86"
server_key_version: "encryptionkey@0"
I20260809 03:35:29.910766 12620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "a4f7a00046b74bffb36c8796ffa67be1"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "0a8cea7f5f53194206f590f8e016b414"
server_key_iv: "ffb7063cd017d7004e5dadbb9bfeed86"
server_key_version: "encryptionkey@0"
I20260809 03:35:29.915028 12620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:29.917678 12644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:29.919025 12620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:35:29.919174 12620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "a4f7a00046b74bffb36c8796ffa67be1"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "0a8cea7f5f53194206f590f8e016b414"
server_key_iv: "ffb7063cd017d7004e5dadbb9bfeed86"
server_key_version: "encryptionkey@0"
I20260809 03:35:29.919297 12620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:29.935904 12620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:29.939003 12620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:29.939251 12620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:29.939905 12620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:29.940848 12620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:29.940930 12620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:29.941006 12620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:29.941058 12620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:29.950500 12620 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:41963
I20260809 03:35:29.950528 12814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:41963 every 8 connection(s)
I20260809 03:35:29.951509 12620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:29.954972 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 12620
I20260809 03:35:29.955133 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:29.955540 1921 external_mini_cluster.cc:1428] Setting key 20a6c055757933682cdfbad2ca3c9e3e
Aug 09 03:35:29 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246529, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:29.967233 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:29.953631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:50829 (local address 127.1.224.126:39163)
0809 03:35:29.953791 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:29.953796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:29.954743 (+ 947us) server_negotiation.cc:407] Connection header received
0809 03:35:29.956504 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:29.956510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:29.956600 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:29.956705 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:29.958310 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.958949 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.960108 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.960325 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.960403 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:29.963212 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:29.963215 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:29.963218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:29.963247 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:29.965172 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:29.965733 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:29.965736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:29.965737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:29.965821 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:29.966167 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:29.966170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:29.966171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:29.966370 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:29.966448 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:29.966980 (+ 532us) server_negotiation.cc:299] Negotiation successful
0809 03:35:29.967098 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:35:29.968261 12822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:29.954040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.67:50829)
0809 03:35:29.954544 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:29.954590 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:29.956228 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:29.956915 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:29.956925 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:29.957426 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:29.958170 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:29.958189 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.959073 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.959076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:29.959999 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:29.960009 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:29.960466 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:29.960469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:29.960681 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:29.961348 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:29.961366 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:29.962936 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:29.965305 (+ 2369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:29.965312 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:29.965329 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:29.965632 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:29.965906 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:29.965908 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:29.965909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:29.966084 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:29.966460 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:29.966466 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:29.966751 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 03:35:29.966929 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 03:35:29.967242 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":130,"threads_started":1}
I20260809 03:35:29.969564 12817 heartbeater.cc:347] Connected to a master server at 127.1.224.126:39163
I20260809 03:35:29.969880 12817 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:29.970458 12817 heartbeater.cc:511] Master 127.1.224.126:39163 requested a full tablet report, sending...
I20260809 03:35:29.971604 12129 ts_manager.cc:194] Registered new tserver with Master: a4f7a00046b74bffb36c8796ffa67be1 (127.1.224.67:41963)
I20260809 03:35:29.972242 12129 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:50829
I20260809 03:35:29.982412 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.224.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.125.keytab.
Entry for principal oryx/127.1.224.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.125.keytab.
I20260809 03:35:30.013091 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.125:38527
--webserver_interface=127.1.224.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.125.keytab
--principal=oryx/127.1.224.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:39163,127.1.224.125:38527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:30.133437 12830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:30.133764 12830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:30.133852 12830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:30.137928 12830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:30.138021 12830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:30.138060 12830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:30.138109 12830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:30.138142 12830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:30.143550 12830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:39163,127.1.224.125:38527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.125.keytab
--principal=oryx/127.1.224.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.125:38527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.224.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
W20260809 03:35:30.143832 12830 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.224.126:39163,127.1.224.125:38527'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 03:35:30.144840 12830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:30.145921 12830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:30.155189 12838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:30.155181 12840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:30.155181 12836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:30.155419 12830 server_base.cc:1034] running on GCE node
I20260809 03:35:30.156140 12830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:30.157487 12830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:30.158690 12830 hybrid_clock.cc:648] HybridClock initialized: now 1786246530158673 us; error 30 us; skew 500 ppm
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:30.161788 12830 init.cc:377] Logged in from keytab as oryx/127.1.224.125@KRBTEST.COM (short username oryx)
I20260809 03:35:30.162822 12830 webserver.cc:460] Webserver started at http://127.1.224.125:46851/ using document root <none> and password file <none>
I20260809 03:35:30.163473 12830 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:30.163573 12830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:30.163839 12830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:30.166219 12830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data/instance:
uuid: "9abcb73ba3c448b79a20e82f3bf2b131"
format_stamp: "Formatted at 2026-08-09 03:35:30 on dist-test-slave-jq4t"
server_key: "6ab6778da3a4801ee20a39a77c78c077"
server_key_iv: "d968337e17c48a1569bdd692d6f8142a"
server_key_version: "encryptionkey@0"
I20260809 03:35:30.166819 12830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/wal/instance:
uuid: "9abcb73ba3c448b79a20e82f3bf2b131"
format_stamp: "Formatted at 2026-08-09 03:35:30 on dist-test-slave-jq4t"
server_key: "6ab6778da3a4801ee20a39a77c78c077"
server_key_iv: "d968337e17c48a1569bdd692d6f8142a"
server_key_version: "encryptionkey@0"
I20260809 03:35:30.170809 12830 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:30.173417 12849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:30.174680 12830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:30.174834 12830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/wal
uuid: "9abcb73ba3c448b79a20e82f3bf2b131"
format_stamp: "Formatted at 2026-08-09 03:35:30 on dist-test-slave-jq4t"
server_key: "6ab6778da3a4801ee20a39a77c78c077"
server_key_iv: "d968337e17c48a1569bdd692d6f8142a"
server_key_version: "encryptionkey@0"
I20260809 03:35:30.174961 12830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:30.187773 12830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:30.191586 12830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:30.191781 12830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:30.199585 12830 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.125:38527
I20260809 03:35:30.199903 12928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.125:38527 every 8 connection(s)
I20260809 03:35:30.200610 12830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
I20260809 03:35:30.201107 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 12830
I20260809 03:35:30.203966 12930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:30.207988 12930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.224.125@KRBTEST.COM
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.125@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:30.217075 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.202824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.1.224.125:38527)
0809 03:35:30.203283 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.203299 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.203338 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:35:30.204059 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.204086 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.204557 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.204920 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.205846 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.206884 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.207773 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.208251 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.208448 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.211730 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.211736 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.211745 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.211773 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.213556 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.214029 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.214035 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.214041 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.214193 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.214546 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.214551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.214553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.215142 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.215340 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.215680 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.215989 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":147,"threads_started":1}
I20260809 03:35:30.220072 12935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.210203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55060)
0809 03:35:30.210649 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.210690 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.210859 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.211224 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.211232 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.211713 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.211846 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.211852 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.212973 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.212975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.213647 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.213659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.214146 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.214149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.214236 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.214803 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.214814 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.216388 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.218354 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.218361 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.218367 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.218659 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.218934 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.218937 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.218939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.219033 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.219526 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.219533 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.219662 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.219778 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.219915 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":177,"threads_started":1}
I20260809 03:35:30.220145 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.210277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55060 (local address 127.1.224.126:39163)
0809 03:35:30.210456 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.210463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.210849 (+ 386us) server_negotiation.cc:407] Connection header received
0809 03:35:30.210979 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.210982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.211057 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.211142 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.212002 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.212854 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.213775 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.214029 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.214141 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.216561 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.216565 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.216568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.216594 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.218241 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.218760 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.218763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.218764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.218840 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.219123 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.219125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.219126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.219408 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.219503 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.219852 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.220011 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 03:35:30.221498 12930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.125@KRBTEST.COM for oryx/127.1.224.125@KRBTEST.COM
I20260809 03:35:30.229811 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.222106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.1.224.125:38527)
0809 03:35:30.222182 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.222185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.222196 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:35:30.222277 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.222279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.222310 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.222367 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.222791 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.223648 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.224449 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.224728 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.224839 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.226808 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.226812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.226821 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.226842 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.228162 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.228609 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.228613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.228615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.228691 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.228970 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.228973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.228975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.229161 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.229268 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.229508 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.229676 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":25}
I20260809 03:35:30.229811 12935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.222009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38527 (local address 127.0.0.1:56524)
0809 03:35:30.222121 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.222136 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.222213 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.222397 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.222399 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.222533 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.222625 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.222651 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.223774 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.223778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.224331 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.224339 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.224839 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.224842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.224931 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.225427 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.225447 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.226679 (+ 1232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.228275 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.228280 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.228282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.228524 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.228790 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.228793 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.228794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.228875 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.229255 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.229258 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.229347 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.229462 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.229617 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 03:35:30.233250 12930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: Bootstrap starting.
I20260809 03:35:30.235707 12930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:30.236532 12930 log.cc:824] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:30.238626 12930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: No bootstrap required, opened a new log
I20260809 03:35:30.241741 12930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } }
I20260809 03:35:30.241955 12930 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:30.242040 12930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9abcb73ba3c448b79a20e82f3bf2b131, State: Initialized, Role: FOLLOWER
I20260809 03:35:30.242518 12930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } }
I20260809 03:35:30.243083 12939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } } }
I20260809 03:35:30.243235 12939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:35:30.243824 12930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 03:35:30.248857 12953 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:35:30.248946 12953 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:35:30.251466 12830 master_runner.cc:428] Detected that this master 9abcb73ba3c448b79a20e82f3bf2b131 is joining an existing cluster
I20260809 03:35:30.251538 12830 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.224.125:38527
I20260809 03:35:30.262898 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.254175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.1.224.126:39163)
0809 03:35:30.254316 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.254321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.254802 (+ 481us) server_negotiation.cc:407] Connection header received
0809 03:35:30.255006 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.255010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.255065 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.255118 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.255776 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.256406 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.257355 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.257701 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.257771 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.259168 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.259172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.259175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.259210 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.261103 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.261676 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.261680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.261681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.261766 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.262052 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.262055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.262057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.262248 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.262357 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.262630 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.262751 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:35:30.262924 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.254062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55066)
0809 03:35:30.254707 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.254722 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.254883 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.255283 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.255286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.255510 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.255642 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.255650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.256555 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.256559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.257196 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.257204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.257915 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.257917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.257990 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.258686 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.258698 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.259022 (+ 324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.261231 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.261234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.261236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.261573 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.261863 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.261867 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.261869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.261956 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.262347 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.262350 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.262457 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.262573 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.262719 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":87,"spinlock_wait_cycles":3968,"thread_start_us":97,"threads_started":1}
I20260809 03:35:30.263167 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.254228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56536 (local address 127.1.224.125:38527)
0809 03:35:30.254415 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.254420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.254634 (+ 214us) server_negotiation.cc:407] Connection header received
0809 03:35:30.254787 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.254790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.254835 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.254915 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.255386 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.256002 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.257330 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.257644 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.257753 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.259725 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.259728 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.259730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.259751 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.261569 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.262020 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.262024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.262025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.262094 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.262354 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.262357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.262359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.262549 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.262622 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.262912 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.263021 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260809 03:35:30.263616 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.254062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38527 (local address 127.0.0.1:56536)
0809 03:35:30.254558 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.254573 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.254687 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.254927 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.254930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.255171 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.255274 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.255279 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.256132 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.256136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.257157 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.257168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.257826 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.257832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.257946 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.259100 (+ 1154us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.259117 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.259532 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.261680 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.261682 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.261685 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.261927 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.262190 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.262193 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.262195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.262273 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.262679 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.262682 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.262754 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.263340 (+ 586us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.263459 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":6,"thread_start_us":126,"threads_started":1}
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246529, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.126@KRBTEST.COM for oryx/127.1.224.125@KRBTEST.COM
I20260809 03:35:30.277734 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.268087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56546 (local address 127.1.224.125:38527)
0809 03:35:30.268210 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.268214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.268530 (+ 316us) server_negotiation.cc:407] Connection header received
0809 03:35:30.268755 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.268758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.268808 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.268898 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.269516 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.270037 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.271154 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.271375 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.271484 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.274155 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.274160 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.274163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.274190 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.275772 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.276259 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.276263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.276265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.276353 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.276708 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.276711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.276713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.276937 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.277038 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.277462 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.277581 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:35:30.277769 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.268048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38527 (local address 127.0.0.1:56546)
0809 03:35:30.268388 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.268430 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.268625 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.268935 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.268940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.269289 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.269393 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.269398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.270161 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.270164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.271031 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.271041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.271495 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.271498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.271609 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.272161 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.272174 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.273999 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.275883 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.275892 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.275898 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.276160 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.276446 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.276448 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.276450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.276600 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.277049 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.277055 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.277208 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.277363 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.277584 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":97,"threads_started":1}
I20260809 03:35:30.279953 12129 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.224.125:38527: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131"
member_type: NON_VOTER
last_known_addr {
host: "127.1.224.125"
port: 38527
}
attrs {
promote: true
}
}
dest_uuid: "418b6f7249104662b195da37a5772666"
cas_config_opid_index: -1
I20260809 03:35:30.281086 12129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.1.224.124@KRBTEST.COM; defaulting to no policy
I20260809 03:35:30.284451 12971 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9abcb73ba3c448b79a20e82f3bf2b131 (127.1.224.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true } } }
I20260809 03:35:30.285475 12889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.124@KRBTEST.COM" created.
I20260809 03:35:30.287215 12889 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 FOLLOWER]: Refusing update from remote peer 418b6f7249104662b195da37a5772666: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 03:35:30.287451 12971 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.224.125:38527
I20260809 03:35:30.287804 12974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true } } }
I20260809 03:35:30.287925 12974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:30.288244 12830 master.cc:537] Master@127.1.224.125:38527 shutting down...
I20260809 03:35:30.288199 12974 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:35:30.290407 12106 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 03:35:30.293346 12106 consensus_peers.cc:487] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 -> Peer 9abcb73ba3c448b79a20e82f3bf2b131 (127.1.224.125:38527): Couldn't send request to peer 9abcb73ba3c448b79a20e82f3bf2b131. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 03:35:30.294137 12830 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:30.294448 12830 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:30.294560 12830 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.124.keytab.
Entry for principal oryx/127.1.224.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.124.keytab.
I20260809 03:35:30.307286 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.124:45879
--webserver_interface=127.1.224.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.124.keytab
--principal=oryx/127.1.224.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:39163,127.1.224.125:38527,127.1.224.124:45879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:30.310710 12830 master.cc:552] Master@127.1.224.125:38527 shutdown complete.
I20260809 03:35:30.310782 12830 master_runner.cc:305] Clearing existing system tablet
I20260809 03:35:30.311101 12830 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:30.311172 12830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:30.312337 12830 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.002s
I20260809 03:35:30.313356 12984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:30.313594 12830 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:30.313747 12830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:30.313786 12830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/wal
uuid: "9abcb73ba3c448b79a20e82f3bf2b131"
format_stamp: "Formatted at 2026-08-09 03:35:30 on dist-test-slave-jq4t"
server_key: "6ab6778da3a4801ee20a39a77c78c077"
server_key_iv: "d968337e17c48a1569bdd692d6f8142a"
server_key_version: "encryptionkey@0"
I20260809 03:35:30.315624 12830 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:30.317657 12830 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 03:35:30.317773 12830 log.cc:1196] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 03:35:30.318027 12830 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: Deleting consensus metadata
I20260809 03:35:30.318222 12830 master_runner.cc:315] Copying system tablet from 127.1.224.126:39163
I20260809 03:35:30.319591 12830 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: tablet copy: Beginning tablet copy session from remote peer at address 127.1.224.126:39163
I20260809 03:35:30.328208 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.320400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55070 (local address 127.1.224.126:39163)
0809 03:35:30.320568 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.320575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.320767 (+ 192us) server_negotiation.cc:407] Connection header received
0809 03:35:30.320932 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.320936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.320984 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.321075 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.321568 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.322131 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.323101 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.323357 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.323474 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.324871 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.324875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.324878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.324906 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.326438 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.326910 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.326914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.326915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.327005 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.327288 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.327293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.327294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.327520 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.327635 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.327912 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.328032 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:35:30.328215 12992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.320292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55070)
0809 03:35:30.320661 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.320679 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.320820 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.321094 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.321096 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.321327 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.321439 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.321445 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.322243 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.322249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.322922 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.322932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.323598 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.323601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.323675 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.324289 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.324302 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.324716 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.326556 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.326559 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.326570 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.326788 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.327100 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.327104 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.327106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.327194 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.327624 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.327628 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.327739 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.327849 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.328017 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":98,"threads_started":1}
I20260809 03:35:30.329267 12166 tablet_copy_service.cc:140] P 418b6f7249104662b195da37a5772666: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9abcb73ba3c448b79a20e82f3bf2b131 ({username='oryx', principal='oryx/127.1.224.125@KRBTEST.COM'} at 127.0.0.1:55070)
I20260809 03:35:30.329492 12166 tablet_copy_service.cc:161] P 418b6f7249104662b195da37a5772666: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9abcb73ba3c448b79a20e82f3bf2b131 at {username='oryx', principal='oryx/127.1.224.125@KRBTEST.COM'} at 127.0.0.1:55070: session id = 9abcb73ba3c448b79a20e82f3bf2b131-00000000000000000000000000000000
I20260809 03:35:30.331194 12166 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 03:35:30.333135 12830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 03:35:30.335258 12830 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: tablet copy: Starting download of 0 data blocks...
I20260809 03:35:30.335487 12830 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: tablet copy: Starting download of 1 WAL segments...
I20260809 03:35:30.337392 12830 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 03:35:30.339099 12166 tablet_copy_service.cc:342] P 418b6f7249104662b195da37a5772666: Request end of tablet copy session 9abcb73ba3c448b79a20e82f3bf2b131-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.224.125@KRBTEST.COM'} at 127.0.0.1:55070
I20260809 03:35:30.339231 12166 tablet_copy_service.cc:433] P 418b6f7249104662b195da37a5772666: ending tablet copy session 9abcb73ba3c448b79a20e82f3bf2b131-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9abcb73ba3c448b79a20e82f3bf2b131
I20260809 03:35:30.346813 12830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:30.348660 12993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:30.348766 12830 server_base.cc:1034] running on GCE node
W20260809 03:35:30.349045 12998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:30.349246 13000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:30.349504 12830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:30.349859 12830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:30.351024 12830 hybrid_clock.cc:648] HybridClock initialized: now 1786246530351009 us; error 34 us; skew 500 ppm
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:30.352912 12830 init.cc:377] Logged in from keytab as oryx/127.1.224.125@KRBTEST.COM (short username oryx)
I20260809 03:35:30.353475 12830 webserver.cc:460] Webserver started at http://127.1.224.125:34313/ using document root <none> and password file <none>
I20260809 03:35:30.353704 12830 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:30.353761 12830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:30.354967 12830 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:30.355921 13013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:30.356173 12830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:30.356266 12830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/wal
uuid: "9abcb73ba3c448b79a20e82f3bf2b131"
format_stamp: "Formatted at 2026-08-09 03:35:30 on dist-test-slave-jq4t"
server_key: "6ab6778da3a4801ee20a39a77c78c077"
server_key_iv: "d968337e17c48a1569bdd692d6f8142a"
server_key_version: "encryptionkey@0"
I20260809 03:35:30.356371 12830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:30.362603 12830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:30.364845 12830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:30.370209 12830 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.125:38527
I20260809 03:35:30.370920 12830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
I20260809 03:35:30.371425 13082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.125:38527 every 8 connection(s)
I20260809 03:35:30.372820 13083 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:35:30.373740 13083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: Bootstrap starting.
I20260809 03:35:30.383126 13083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:35:30.383505 13083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: Bootstrap complete.
I20260809 03:35:30.384081 13083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true } }
I20260809 03:35:30.384228 13083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9abcb73ba3c448b79a20e82f3bf2b131, State: Initialized, Role: LEARNER
I20260809 03:35:30.384353 13083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true } }
I20260809 03:35:30.384765 13090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true } } }
I20260809 03:35:30.384891 13083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:30.384963 13090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: This master's current role is: LEARNER
I20260809 03:35:30.388010 13112 catalog_manager.cc:1259] Loaded cluster ID: 83dc0a9e3c9e46ad9d196726bcc3072b
I20260809 03:35:30.388130 13112 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: loading cluster ID for follower catalog manager: success
I20260809 03:35:30.390013 13112 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: acquiring CA information for follower catalog manager: success
I20260809 03:35:30.391225 13112 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 03:35:30.463289 12982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:30.463572 12982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:30.463613 12982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:30.467630 12982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:30.467690 12982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:30.467710 12982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:30.467726 12982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:30.467783 12982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:30.473294 12982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:39163,127.1.224.125:38527,127.1.224.124:45879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.124.keytab
--principal=oryx/127.1.224.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.124:45879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.224.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:30.474823 12982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:30.476325 12982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:30.488858 13121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:30.488858 13127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:30.489398 12982 server_base.cc:1034] running on GCE node
W20260809 03:35:30.490693 13129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:30.491441 12982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:30.492830 12982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:30.494021 12982 hybrid_clock.cc:648] HybridClock initialized: now 1786246530494007 us; error 33 us; skew 500 ppm
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:30.497205 12982 init.cc:377] Logged in from keytab as oryx/127.1.224.124@KRBTEST.COM (short username oryx)
I20260809 03:35:30.498200 12982 webserver.cc:460] Webserver started at http://127.1.224.124:34299/ using document root <none> and password file <none>
I20260809 03:35:30.498817 12982 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:30.498917 12982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:30.499171 12982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:30.501456 12982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data/instance:
uuid: "8ff6a3b7018b4fac9abff079bdbae592"
format_stamp: "Formatted at 2026-08-09 03:35:30 on dist-test-slave-jq4t"
server_key: "cea6a4da4705ab4bc34031965a1c2621"
server_key_iv: "a610f1c059e84946b911d753bd953b57"
server_key_version: "encryptionkey@0"
I20260809 03:35:30.502080 12982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/wal/instance:
uuid: "8ff6a3b7018b4fac9abff079bdbae592"
format_stamp: "Formatted at 2026-08-09 03:35:30 on dist-test-slave-jq4t"
server_key: "cea6a4da4705ab4bc34031965a1c2621"
server_key_iv: "a610f1c059e84946b911d753bd953b57"
server_key_version: "encryptionkey@0"
I20260809 03:35:30.506047 12982 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:30.508685 13139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:30.510062 12982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:30.510214 12982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/wal
uuid: "8ff6a3b7018b4fac9abff079bdbae592"
format_stamp: "Formatted at 2026-08-09 03:35:30 on dist-test-slave-jq4t"
server_key: "cea6a4da4705ab4bc34031965a1c2621"
server_key_iv: "a610f1c059e84946b911d753bd953b57"
server_key_version: "encryptionkey@0"
I20260809 03:35:30.510334 12982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:30.513494 12414 heartbeater.cc:503] Master 127.1.224.126:39163 was elected leader, sending a full tablet report...
I20260809 03:35:30.531224 12982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:30.534608 12982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:30.534837 12982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:30.542127 12982 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.124:45879
I20260809 03:35:30.542459 13212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.124:45879 every 8 connection(s)
I20260809 03:35:30.543218 12982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
I20260809 03:35:30.546350 13213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:30.547554 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 12982
I20260809 03:35:30.552388 13213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.224.124@KRBTEST.COM
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.124@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:30.563616 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.549174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.1.224.124:45879)
0809 03:35:30.549580 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.549595 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.549632 (+ 37us) server_negotiation.cc:407] Connection header received
0809 03:35:30.550638 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.550669 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.551120 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.551525 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.552449 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.553786 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.555044 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.555536 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.555713 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.558533 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.558540 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.558555 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.558590 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.560434 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.560925 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.560930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.560935 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.561045 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.561329 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.561333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.561335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.561733 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.561985 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.562269 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.562591 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":108,"threads_started":1}
I20260809 03:35:30.563648 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.554374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55078 (local address 127.1.224.126:39163)
0809 03:35:30.554529 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.554534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.554763 (+ 229us) server_negotiation.cc:407] Connection header received
0809 03:35:30.554962 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.554966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.555023 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.555110 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.555783 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.556299 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.557404 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.557823 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.557927 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.560270 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.560276 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.560281 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.560314 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.561856 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.562324 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.562329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.562331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.562437 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.562743 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.562746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.562747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.562921 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.562997 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.563408 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.563517 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 03:35:30.563658 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.554228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55078)
0809 03:35:30.554647 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.554686 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.554818 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.555160 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.555166 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.555529 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.555653 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.555658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.556449 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.556453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.557280 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.557291 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.557966 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.557970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.558092 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.558737 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.558750 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.560120 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.561974 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.561984 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.561990 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.562218 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.562546 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.562552 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.562554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.562647 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.563027 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.563033 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.563152 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.563305 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.563484 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":156,"threads_started":1}
I20260809 03:35:30.565006 13213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.124@KRBTEST.COM for oryx/127.1.224.125@KRBTEST.COM
I20260809 03:35:30.573042 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.565502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38527 (local address 127.0.0.1:56556)
0809 03:35:30.565621 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.565635 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.565732 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.566156 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.566158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.566285 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.566377 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.566382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.567251 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.567254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.567771 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.567778 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.568197 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.568200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.568272 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.568803 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.568815 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.570034 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.571676 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.571679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.571681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.571872 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.572121 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.572124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.572125 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.572200 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.572611 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.572614 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.572707 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.572791 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.572899 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 03:35:30.573135 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.565641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56556 (local address 127.1.224.125:38527)
0809 03:35:30.565939 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.565943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.565957 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:35:30.566000 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.566002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.566041 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.566120 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.566484 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.567159 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.567883 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.568107 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.568182 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.570179 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.570182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.570184 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.570211 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.571574 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.571960 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.571963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.571964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.572041 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.572278 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.572280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.572281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.572467 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.572606 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.572852 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.572965 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":102,"threads_started":1}
I20260809 03:35:30.573900 13213 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.124@KRBTEST.COM for oryx/127.1.224.124@KRBTEST.COM
I20260809 03:35:30.582252 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.574505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:45879 (local address 127.0.0.1:55198)
0809 03:35:30.574642 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.574655 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.574720 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.574927 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.574929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.575069 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.575145 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.575150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.576221 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.576223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.576802 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.576810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.577239 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.577241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.577298 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.577745 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.577757 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.578977 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.580879 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.580882 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.580883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.581076 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.581388 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.581390 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.581392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.581478 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.581834 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.581836 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.581913 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.582008 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.582113 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 03:35:30.582285 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.574564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55198 (local address 127.1.224.124:45879)
0809 03:35:30.574693 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.574697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.574710 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:30.574795 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.574797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.574832 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.574902 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.575242 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.576118 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.576911 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.577142 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.577224 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.579111 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.579114 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.579116 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.579138 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.580716 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.581191 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.581194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.581195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.581274 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.581566 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.581568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.581569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.581750 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.581827 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.582072 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.582173 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:35:30.585574 13213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: Bootstrap starting.
I20260809 03:35:30.588330 13213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:30.589172 13213 log.cc:824] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:30.591239 13213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: No bootstrap required, opened a new log
I20260809 03:35:30.594465 13213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } }
I20260809 03:35:30.594691 13213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:30.594772 13213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ff6a3b7018b4fac9abff079bdbae592, State: Initialized, Role: FOLLOWER
I20260809 03:35:30.595211 13213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } }
I20260809 03:35:30.595736 13219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } } }
I20260809 03:35:30.595881 13219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:35:30.596637 13213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 03:35:30.602355 13243 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:35:30.602458 13243 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:35:30.606570 12982 master_runner.cc:428] Detected that this master 8ff6a3b7018b4fac9abff079bdbae592 is joining an existing cluster
I20260809 03:35:30.606645 12982 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.224.124:45879
I20260809 03:35:30.617167 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 12222
I20260809 03:35:30.646085 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.609752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55204 (local address 127.1.224.124:45879)
0809 03:35:30.609873 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.609878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.610199 (+ 321us) server_negotiation.cc:407] Connection header received
0809 03:35:30.610641 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.610647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.610695 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.610771 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.611383 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.612151 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.613428 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.613754 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.613828 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.642438 (+ 28610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.642443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.642446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.642482 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.644268 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.644781 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.644785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.644787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.644860 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.645111 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.645114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.645115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.645355 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.645439 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.645758 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.645890 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:35:30.646266 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:33961
--local_ip_for_outbound_sockets=127.1.224.65
--tserver_master_addrs=127.1.224.126:39163
--webserver_port=39417
--webserver_interface=127.1.224.65
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab
--principal=oryx/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.224.126:39163,127.1.224.125:38527
--tserver_master_addrs=127.1.224.126:39163,127.1.224.125:38527,127.1.224.124:45879 with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:30.646862 13255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.609351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:45879 (local address 127.0.0.1:55204)
0809 03:35:30.610123 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.610140 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.610237 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.610894 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.610897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.611120 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.611245 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.611253 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.612274 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.612277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.613291 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.613303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.614031 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.614034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.614126 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.614726 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.614743 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.642234 (+ 27491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.644395 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.644398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.644400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.644678 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.644949 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.644952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.644954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.645029 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.645494 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.645496 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.645589 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.646522 (+ 933us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.646675 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":1,"thread_start_us":124,"threads_started":1}
I20260809 03:35:30.649416 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.609364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55094 (local address 127.1.224.126:39163)
0809 03:35:30.609516 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.609522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.610560 (+ 1038us) server_negotiation.cc:407] Connection header received
0809 03:35:30.610745 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.610752 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.610810 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.610889 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.611594 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.612347 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.613621 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.613946 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.614028 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.642535 (+ 28507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.642543 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.642549 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.642594 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.645075 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.645657 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.645663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.645668 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.645767 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.647337 (+ 1570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.647344 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.647348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.647646 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.647754 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.649111 (+ 1357us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.649255 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:35:30.649437 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.609694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55094)
0809 03:35:30.610453 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.610465 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.610628 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.611106 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.611111 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.611300 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.611444 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.611451 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.612513 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.612516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.613492 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.613503 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.614191 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.614195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.614730 (+ 535us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.615228 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.615239 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.642217 (+ 26978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.645236 (+ 3019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.645242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.645244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.645532 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.645888 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.645891 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.645893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.646405 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.648762 (+ 2357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.648767 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.648893 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.649014 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.649187 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":504,"thread_start_us":143,"threads_started":1}
I20260809 03:35:30.651837 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.609844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56558 (local address 127.1.224.125:38527)
0809 03:35:30.609980 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.609984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.610220 (+ 236us) server_negotiation.cc:407] Connection header received
0809 03:35:30.610337 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.610342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.610415 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.610508 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.611196 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.612054 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.613593 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.613947 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.614028 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.646158 (+ 32130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.646166 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.646173 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.646211 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.648634 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.650164 (+ 1530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.650171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.650175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.650278 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.650631 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.650638 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.650643 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.650941 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.651025 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.651447 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.651626 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:35:30.653755 13252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.609313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38527 (local address 127.0.0.1:56558)
0809 03:35:30.610102 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.610118 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.610237 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.610553 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.610557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.610897 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.611056 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.611065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.612194 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.612197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.613435 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.613447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.614191 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.614195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.614371 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.641243 (+ 26872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.641267 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.645960 (+ 4693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.649751 (+ 3791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.649757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.649762 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.650031 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.650414 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.650419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.650424 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.650520 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.651106 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.651112 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.651224 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.652417 (+ 1193us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.652652 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":214,"thread_start_us":117,"threads_started":1}
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246529, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.126@KRBTEST.COM for oryx/127.1.224.124@KRBTEST.COM
I20260809 03:35:30.663143 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.654622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55206 (local address 127.1.224.124:45879)
0809 03:35:30.654733 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.654737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.654750 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:30.654945 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.654947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.654996 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.655103 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.655542 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.656067 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.657095 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.657430 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.657558 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.659810 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.659814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.659816 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.659844 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.661417 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.661885 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.661889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.661890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.662008 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.662281 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.662286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.662288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.662480 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.662587 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.662869 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.662976 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:35:30.663143 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.654546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:45879 (local address 127.0.0.1:55206)
0809 03:35:30.654678 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.654696 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.654807 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.655126 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.655129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.655326 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.655421 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.655427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.656212 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.656216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.656940 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.656957 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.657563 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.657567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.657662 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.658244 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.658259 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.659670 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.661563 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.661566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.661568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.661775 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.662088 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.662091 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.662093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.662185 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.662592 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.662596 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.662702 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.662813 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.662978 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 03:35:30.664433 12129 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.224.124:45879: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592"
member_type: NON_VOTER
last_known_addr {
host: "127.1.224.124"
port: 45879
}
attrs {
promote: true
}
}
dest_uuid: "418b6f7249104662b195da37a5772666"
cas_config_opid_index: 5
I20260809 03:35:30.665246 12129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } }
I20260809 03:35:30.666472 12974 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8ff6a3b7018b4fac9abff079bdbae592 (127.1.224.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:30.667701 13177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:30.668162 12974 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.224.124:45879
I20260809 03:35:30.668354 12974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:30.668470 12974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:30.669099 12982 master.cc:537] Master@127.1.224.124:45879 shutting down...
I20260809 03:35:30.669317 13177 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 1 FOLLOWER]: Refusing update from remote peer 418b6f7249104662b195da37a5772666: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 03:35:30.670822 12971 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:35:30.672928 12107 consensus_peers.cc:487] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 -> Peer 8ff6a3b7018b4fac9abff079bdbae592 (127.1.224.124:45879): Couldn't send request to peer 8ff6a3b7018b4fac9abff079bdbae592. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 03:35:30.675500 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.666894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56574 (local address 127.1.224.125:38527)
0809 03:35:30.667046 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.667050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.670199 (+ 3149us) server_negotiation.cc:407] Connection header received
0809 03:35:30.670397 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.670402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.670480 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.670563 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:30.671123 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.671882 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.673574 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.674244 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.674327 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.674495 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.675252 (+ 757us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.675357 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:35:30.676201 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.666648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38527 (local address 127.0.0.1:56574)
0809 03:35:30.670122 (+ 3474us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.670137 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.670271 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.670822 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.670825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.670866 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:30.670972 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.670979 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.672038 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.672041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.673410 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.673425 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.674973 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.674976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.675069 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.675116 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.675959 (+ 843us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.676058 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3404}
I20260809 03:35:30.676599 12982 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:30.676856 13046 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 LEARNER]: Refusing update from remote peer 418b6f7249104662b195da37a5772666: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 03:35:30.677032 12982 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:30.677204 12982 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: stopping tablet replica
I20260809 03:35:30.677340 12971 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 03:35:30.679390 13046 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8ff6a3b7018b4fac9abff079bdbae592 (127.1.224.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:30.681243 13259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:30.681471 13259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: This master's current role is: LEARNER
I20260809 03:35:30.681754 13090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 418b6f7249104662b195da37a5772666. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: NON_VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: true } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:30.681861 13090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: This master's current role is: LEARNER
I20260809 03:35:30.690083 12971 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666: attempting to promote NON_VOTER 9abcb73ba3c448b79a20e82f3bf2b131 to VOTER
I20260809 03:35:30.690584 12971 consensus_queue.cc:237] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } }
I20260809 03:35:30.692386 13046 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 LEARNER]: Refusing update from remote peer 418b6f7249104662b195da37a5772666: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 03:35:30.692793 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.691821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:45879 (local address 127.0.0.1:55220)
0809 03:35:30.692164 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.692705 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:45879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225}
I20260809 03:35:30.692982 13260 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 03:35:30.695078 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.693354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:45879 (local address 127.0.0.1:55228)
0809 03:35:30.694935 (+ 1581us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.694999 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:45879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":87,"threads_started":1}
W20260809 03:35:30.695169 12107 consensus_peers.cc:487] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 -> Peer 8ff6a3b7018b4fac9abff079bdbae592 (127.1.224.124:45879): Couldn't send request to peer 8ff6a3b7018b4fac9abff079bdbae592. Status: Network error: Client connection negotiation failed: client connection to 127.1.224.124:45879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 03:35:30.695423 12982 master.cc:552] Master@127.1.224.124:45879 shutdown complete.
I20260809 03:35:30.695482 12982 master_runner.cc:305] Clearing existing system tablet
I20260809 03:35:30.695488 12971 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9abcb73ba3c448b79a20e82f3bf2b131 (127.1.224.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:30.695750 12982 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:30.695806 12982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:30.697295 12971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:30.697432 12971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:30.698004 12974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:30.698127 12974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:30.698480 13046 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9abcb73ba3c448b79a20e82f3bf2b131 (127.1.224.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:30.700001 13090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:30.700129 13090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:35:30.700896 12982 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.000s
I20260809 03:35:30.701876 13265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:30.702102 12982 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:30.702261 12982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:30.702301 12982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/wal
uuid: "8ff6a3b7018b4fac9abff079bdbae592"
format_stamp: "Formatted at 2026-08-09 03:35:30 on dist-test-slave-jq4t"
server_key: "cea6a4da4705ab4bc34031965a1c2621"
server_key_iv: "a610f1c059e84946b911d753bd953b57"
server_key_version: "encryptionkey@0"
I20260809 03:35:30.704245 12982 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:30.706506 12982 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 03:35:30.706624 12982 log.cc:1196] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 03:35:30.706929 12982 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: Deleting consensus metadata
I20260809 03:35:30.707118 12982 master_runner.cc:315] Copying system tablet from 127.1.224.126:39163
I20260809 03:35:30.708395 12982 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: tablet copy: Beginning tablet copy session from remote peer at address 127.1.224.126:39163
I20260809 03:35:30.717981 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.709156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55108 (local address 127.1.224.126:39163)
0809 03:35:30.709314 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.709322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.709618 (+ 296us) server_negotiation.cc:407] Connection header received
0809 03:35:30.709793 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.709796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.709854 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.709935 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.710687 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.711441 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.712332 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.712698 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.712800 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.714141 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.714145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.714148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.714185 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.715764 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.716245 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.716249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.716251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.716343 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.716721 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.716725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.716727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.717034 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.717158 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.717674 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.717810 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 03:35:30.718004 13287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.709084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55108)
0809 03:35:30.709505 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.709519 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.709670 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.710093 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.710095 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.710346 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.710551 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.710559 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.711565 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.711569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.712185 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.712195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.712821 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.712825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.712930 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.713591 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.713605 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.713971 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.715891 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.715896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.715898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.716133 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.716443 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.716447 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.716449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.716612 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.717158 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.717161 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.717326 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.717454 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.717752 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":98,"threads_started":1}
I20260809 03:35:30.718884 12166 tablet_copy_service.cc:140] P 418b6f7249104662b195da37a5772666: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8ff6a3b7018b4fac9abff079bdbae592 ({username='oryx', principal='oryx/127.1.224.124@KRBTEST.COM'} at 127.0.0.1:55108)
I20260809 03:35:30.719017 12166 tablet_copy_service.cc:161] P 418b6f7249104662b195da37a5772666: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8ff6a3b7018b4fac9abff079bdbae592 at {username='oryx', principal='oryx/127.1.224.124@KRBTEST.COM'} at 127.0.0.1:55108: session id = 8ff6a3b7018b4fac9abff079bdbae592-00000000000000000000000000000000
I20260809 03:35:30.719935 12166 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 03:35:30.721349 12982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 03:35:30.723639 12982 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: tablet copy: Starting download of 0 data blocks...
I20260809 03:35:30.723863 12982 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: tablet copy: Starting download of 1 WAL segments...
I20260809 03:35:30.725587 12982 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 03:35:30.727255 12166 tablet_copy_service.cc:342] P 418b6f7249104662b195da37a5772666: Request end of tablet copy session 8ff6a3b7018b4fac9abff079bdbae592-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.224.124@KRBTEST.COM'} at 127.0.0.1:55108
I20260809 03:35:30.727363 12166 tablet_copy_service.cc:433] P 418b6f7249104662b195da37a5772666: ending tablet copy session 8ff6a3b7018b4fac9abff079bdbae592-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8ff6a3b7018b4fac9abff079bdbae592
I20260809 03:35:30.734423 12982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:30.736063 13288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:30.736340 13291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:30.736649 12982 server_base.cc:1034] running on GCE node
W20260809 03:35:30.740681 13289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:30.740957 12982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:30.741321 12982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:30.742478 12982 hybrid_clock.cc:648] HybridClock initialized: now 1786246530742466 us; error 28 us; skew 500 ppm
I20260809 03:35:30.742689 12608 heartbeater.cc:503] Master 127.1.224.126:39163 was elected leader, sending a full tablet report...
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:30.744822 12982 init.cc:377] Logged in from keytab as oryx/127.1.224.124@KRBTEST.COM (short username oryx)
I20260809 03:35:30.745340 12982 webserver.cc:460] Webserver started at http://127.1.224.124:36701/ using document root <none> and password file <none>
I20260809 03:35:30.745566 12982 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:30.745623 12982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:30.746798 12982 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:30.747776 13304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:30.748015 12982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:30.748093 12982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/wal
uuid: "8ff6a3b7018b4fac9abff079bdbae592"
format_stamp: "Formatted at 2026-08-09 03:35:30 on dist-test-slave-jq4t"
server_key: "cea6a4da4705ab4bc34031965a1c2621"
server_key_iv: "a610f1c059e84946b911d753bd953b57"
server_key_version: "encryptionkey@0"
I20260809 03:35:30.748185 12982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:30.757472 12982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:30.759723 12982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:30.765131 12982 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.124:45879
I20260809 03:35:30.765136 13384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.124:45879 every 8 connection(s)
I20260809 03:35:30.765763 12982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
I20260809 03:35:30.767282 13385 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:35:30.768175 13385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: Bootstrap starting.
I20260809 03:35:30.777431 13385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:35:30.777770 13385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: Bootstrap complete.
I20260809 03:35:30.778324 13385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } }
I20260809 03:35:30.778476 13385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8ff6a3b7018b4fac9abff079bdbae592, State: Initialized, Role: LEARNER
I20260809 03:35:30.778599 13385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } }
I20260809 03:35:30.778962 13391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:30.779134 13385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:30.779138 13391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: This master's current role is: LEARNER
I20260809 03:35:30.782596 13408 catalog_manager.cc:1259] Loaded cluster ID: 83dc0a9e3c9e46ad9d196726bcc3072b
I20260809 03:35:30.782713 13408 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: loading cluster ID for follower catalog manager: success
I20260809 03:35:30.784817 13408 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: acquiring CA information for follower catalog manager: success
I20260809 03:35:30.786167 13408 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 03:35:30.823165 13256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:30.823560 13256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:30.823627 13256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:30.830188 13256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:30.830289 13256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:30.830386 13256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:30.838590 13256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab
--principal=oryx/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:33961
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=39417
--tserver_master_addrs=127.1.224.126:39163,127.1.224.125:38527,127.1.224.124:45879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:30.840149 13256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:30.841603 13256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:30.853463 13421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:30.853441 13418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:30.853443 13414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:30.853848 13256 server_base.cc:1034] running on GCE node
I20260809 03:35:30.854477 13256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:30.855221 13256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:30.856431 13256 hybrid_clock.cc:648] HybridClock initialized: now 1786246530856394 us; error 52 us; skew 500 ppm
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:30.859540 13256 init.cc:377] Logged in from keytab as oryx/127.1.224.65@KRBTEST.COM (short username oryx)
I20260809 03:35:30.860631 13256 webserver.cc:460] Webserver started at http://127.1.224.65:39417/ using document root <none> and password file <none>
I20260809 03:35:30.861251 13256 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:30.861369 13256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:30.865579 13256 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:35:30.867954 13429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:30.869421 13256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:30.869576 13256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "bca54a7b094841a5883552c27c0e94c0"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "8e445c460699c9a3adb727c44405e4d6"
server_key_iv: "dae532e132148867a04312540f886e02"
server_key_version: "encryptionkey@0"
I20260809 03:35:30.870206 13256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:30.906240 13256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:30.909385 13256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:30.909607 13256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:30.910259 13256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:30.911244 13256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:30.911319 13256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:30.911388 13256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:30.911437 13256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:30.921976 13256 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:33961
I20260809 03:35:30.922006 13608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:33961 every 8 connection(s)
I20260809 03:35:30.923120 13256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:30.926973 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 13256
I20260809 03:35:30.927204 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 12421
I20260809 03:35:30.952165 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:46675
--local_ip_for_outbound_sockets=127.1.224.66
--tserver_master_addrs=127.1.224.126:39163
--webserver_port=33851
--webserver_interface=127.1.224.66
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab
--principal=oryx/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.224.126:39163,127.1.224.125:38527
--tserver_master_addrs=127.1.224.126:39163,127.1.224.125:38527,127.1.224.124:45879 with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.124@KRBTEST.COM
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.125@KRBTEST.COM
Aug 09 03:35:30 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246530, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:30.959226 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.925229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:35799 (local address 127.1.224.124:45879)
0809 03:35:30.925578 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.925595 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.926938 (+ 1343us) server_negotiation.cc:407] Connection header received
0809 03:35:30.944436 (+ 17498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.944441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.944557 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.944622 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.947004 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.948094 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.949418 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.949905 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.950051 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.953736 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.953742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.953746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.953787 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.956859 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.957428 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.957431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.957433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.957532 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.957910 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.957913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.957915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.958198 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.958286 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.958935 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.959074 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
I20260809 03:35:30.959224 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.925553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:34725 (local address 127.1.224.125:38527)
0809 03:35:30.925689 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.925694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.926707 (+ 1013us) server_negotiation.cc:407] Connection header received
0809 03:35:30.944920 (+ 18213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.944924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.945000 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.945087 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.946992 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.947877 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.950375 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.950692 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.950768 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.954368 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.954374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.954377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.954416 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.956280 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.957135 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.957139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.957141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.957233 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.957666 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.957669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.957671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.957966 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.958052 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.958891 (+ 839us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.959049 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:35:30.961231 13617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.925734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:45879 (local address 127.1.224.65:35799)
0809 03:35:30.926819 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.926835 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.944055 (+ 17220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.945071 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.945084 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.945869 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.946793 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.946814 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.948237 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.948240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.949274 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.949284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.950377 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.950385 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.950617 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.951401 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.951422 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.953505 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.956997 (+ 3492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.957000 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.957003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.957311 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.957654 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.957656 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.957658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.957795 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.958339 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.958342 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.958538 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.959466 (+ 928us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.959993 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":340,"spinlock_wait_cycles":3072,"thread_start_us":148,"threads_started":1}
I20260809 03:35:30.961526 13615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.925733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38527 (local address 127.1.224.65:34725)
0809 03:35:30.926553 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.926609 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.944043 (+ 17434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.945674 (+ 1631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.945680 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.945908 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.946789 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.946815 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.948057 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.948063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.950229 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.950240 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.950955 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.950958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.951063 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.952617 (+ 1554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.952633 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.954211 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.956430 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.956438 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.956454 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.957016 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.957347 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.957353 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.957356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.957549 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.958185 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.958194 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.958538 (+ 344us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.959456 (+ 918us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.960047 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":1399,"spinlock_wait_cycles":1792,"thread_start_us":176,"threads_started":1}
I20260809 03:35:30.962853 13611 heartbeater.cc:347] Connected to a master server at 127.1.224.125:38527
I20260809 03:35:30.963138 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.926157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:58767 (local address 127.1.224.126:39163)
0809 03:35:30.926283 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:30.926288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:30.927248 (+ 960us) server_negotiation.cc:407] Connection header received
0809 03:35:30.944683 (+ 17435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:30.944690 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:30.944768 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:30.944828 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:30.947003 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.947837 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.955769 (+ 7932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.956027 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.956161 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.958847 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:30.958853 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:30.958858 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:30.958898 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:30.960912 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.961501 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.961505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.961507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.961613 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:30.961948 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:30.961954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:30.961958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:30.962241 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:30.962340 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:30.962779 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 03:35:30.962933 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:35:30.963155 13611 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:30.963716 13611 heartbeater.cc:511] Master 127.1.224.125:38527 requested a full tablet report, sending...
I20260809 03:35:30.963775 13616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:30.925994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.65:58767)
0809 03:35:30.927149 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:30.927164 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:30.944531 (+ 17367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:30.945070 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:30.945086 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:30.945869 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:30.946791 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.946814 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.948026 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.948029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.955569 (+ 7540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:30.955583 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:30.956165 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:30.956169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:30.956292 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:30.956868 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:30.956888 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:30.958648 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:30.961067 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.961073 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.961078 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.961365 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.961740 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:30.961743 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:30.961745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:30.961830 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:30.962412 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:30.962418 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:30.962547 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 03:35:30.963314 (+ 767us) client_negotiation.cc:240] Negotiation successful
0809 03:35:30.963501 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"mutex_wait_us":1150,"spinlock_wait_cycles":14720,"thread_start_us":103,"threads_started":1}
I20260809 03:35:30.962853 13609 heartbeater.cc:347] Connected to a master server at 127.1.224.124:45879
I20260809 03:35:30.964354 13609 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:30.964630 13610 heartbeater.cc:347] Connected to a master server at 127.1.224.126:39163
I20260809 03:35:30.964666 13609 heartbeater.cc:511] Master 127.1.224.124:45879 requested a full tablet report, sending...
I20260809 03:35:30.964715 13610 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:30.964867 13610 heartbeater.cc:511] Master 127.1.224.126:39163 requested a full tablet report, sending...
I20260809 03:35:30.965673 12129 ts_manager.cc:194] Re-registered known tserver with Master: bca54a7b094841a5883552c27c0e94c0 (127.1.224.65:33961)
I20260809 03:35:30.965692 13024 ts_manager.cc:194] Registered new tserver with Master: bca54a7b094841a5883552c27c0e94c0 (127.1.224.65:33961)
I20260809 03:35:30.965760 13331 ts_manager.cc:194] Registered new tserver with Master: bca54a7b094841a5883552c27c0e94c0 (127.1.224.65:33961)
I20260809 03:35:30.966504 12129 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:58767
I20260809 03:35:30.974474 12817 heartbeater.cc:503] Master 127.1.224.126:39163 was elected leader, sending a full tablet report...
W20260809 03:35:31.078580 13619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:31.079016 13619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:31.079113 13619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:31.084892 13619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:31.084976 13619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:31.085064 13619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:31.091486 13619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab
--principal=oryx/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:46675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=33851
--tserver_master_addrs=127.1.224.126:39163,127.1.224.125:38527,127.1.224.124:45879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:31.092814 13619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:31.093912 13619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:31.104265 13629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:31.104275 13636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:31.104326 13638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:31.104434 13619 server_base.cc:1034] running on GCE node
I20260809 03:35:31.105036 13619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:31.105690 13619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:31.106906 13619 hybrid_clock.cc:648] HybridClock initialized: now 1786246531106886 us; error 30 us; skew 500 ppm
Aug 09 03:35:31 dist-test-slave-jq4t krb5kdc[12065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246531, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:31.109997 13619 init.cc:377] Logged in from keytab as oryx/127.1.224.66@KRBTEST.COM (short username oryx)
I20260809 03:35:31.110934 13619 webserver.cc:460] Webserver started at http://127.1.224.66:33851/ using document root <none> and password file <none>
I20260809 03:35:31.111541 13619 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:31.111639 13619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:31.115609 13619 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:31.117914 13646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:31.119166 13619 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:35:31.119293 13619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "eab5b9e49845493fb3bab869bc385008"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "69acf224b60cd25153e76765c12be8f9"
server_key_iv: "2754c2e9ed3f7e68a56c85bf4e842a48"
server_key_version: "encryptionkey@0"
I20260809 03:35:31.119855 13619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:31.160018 13619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:31.163689 13619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:31.163933 13619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:31.164647 13619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:31.165603 13619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:31.165650 13619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:31.165722 13619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:31.165752 13619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:31.174917 13619 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:46675
I20260809 03:35:31.175040 13818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:46675 every 8 connection(s)
I20260809 03:35:31.176059 13619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:31.179836 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 13619
I20260809 03:35:31.180035 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 12620
Aug 09 03:35:31 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246531, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.125@KRBTEST.COM
Aug 09 03:35:31 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246531, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.124@KRBTEST.COM
I20260809 03:35:31.208272 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.178755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:45711 (local address 127.1.224.125:38527)
0809 03:35:31.178914 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.178920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.179858 (+ 938us) server_negotiation.cc:407] Connection header received
0809 03:35:31.183590 (+ 3732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.183596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.183664 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.183717 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.187629 (+ 3912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.188448 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.189639 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.190000 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.190081 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.194796 (+ 4715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.194803 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.194808 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.194844 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.196645 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.205691 (+ 9046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.205697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.205701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.205802 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.207215 (+ 1413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.207221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.207225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.207480 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.207559 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.207880 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.208039 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 03:35:31.209591 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.178141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:41555 (local address 127.1.224.124:45879)
0809 03:35:31.178423 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.178428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.179346 (+ 918us) server_negotiation.cc:407] Connection header received
0809 03:35:31.185244 (+ 5898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.185250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.185317 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.185369 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.186575 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.187338 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.190979 (+ 3641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.191391 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.191489 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.201791 (+ 10302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.201798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.201803 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.201841 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.203734 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.205049 (+ 1315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.205055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.205059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.205158 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.206104 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.206110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.206114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.206393 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.206475 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.209202 (+ 2727us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.209379 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189}
Aug 09 03:35:31 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246531, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:31.212319 13826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.178590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:45879 (local address 127.1.224.66:41555)
0809 03:35:31.179180 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:31.179234 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:31.183474 (+ 4240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:31.185548 (+ 2074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:31.185553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:31.185768 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:31.186434 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.186441 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.187526 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.187530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.189039 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.189050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.193714 (+ 4664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.193720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.193843 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.196788 (+ 2945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:31.196807 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:31.201556 (+ 4749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:31.203944 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:31.203965 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:31.203995 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:31.204923 (+ 928us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:31.205324 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:31.205329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:31.205334 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:31.205560 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:31.206549 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:31.206559 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:31.206943 (+ 384us) client_negotiation.cc:769] Sending connection context
0809 03:35:31.209989 (+ 3046us) client_negotiation.cc:240] Negotiation successful
0809 03:35:31.211389 (+ 1400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":526,"thread_start_us":134,"threads_started":1}
I20260809 03:35:31.214134 13819 heartbeater.cc:347] Connected to a master server at 127.1.224.124:45879
I20260809 03:35:31.214455 13819 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:31.215173 13819 heartbeater.cc:511] Master 127.1.224.124:45879 requested a full tablet report, sending...
I20260809 03:35:31.216559 13331 ts_manager.cc:194] Registered new tserver with Master: eab5b9e49845493fb3bab869bc385008 (127.1.224.66:46675)
I20260809 03:35:31.217048 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.178590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38527 (local address 127.1.224.66:45711)
0809 03:35:31.179755 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:31.179768 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:31.183402 (+ 3634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:31.184131 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:31.184142 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:31.184692 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:31.186119 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.186141 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.188629 (+ 2488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.188632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.189503 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.189513 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.190302 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.190306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.190514 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.191705 (+ 1191us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:31.191733 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:31.194540 (+ 2807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:31.204253 (+ 9713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:31.204258 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:31.204262 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:31.204690 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:31.205905 (+ 1215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:31.205910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:31.205915 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:31.206004 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:31.207617 (+ 1613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:31.207622 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:31.207715 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 03:35:31.208726 (+ 1011us) client_negotiation.cc:240] Negotiation successful
0809 03:35:31.210452 (+ 1726us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1066,"mutex_wait_us":156,"spinlock_wait_cycles":17024,"thread_start_us":164,"threads_started":1}
I20260809 03:35:31.217886 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.178281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:52069 (local address 127.1.224.126:39163)
0809 03:35:31.178544 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.178550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.179591 (+ 1041us) server_negotiation.cc:407] Connection header received
0809 03:35:31.185590 (+ 5999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.185596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.185667 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.185727 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.186399 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.187295 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.190084 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.190393 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.190485 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.213121 (+ 22636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.213126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.213129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.213165 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.215154 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.215856 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.215860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.215862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.215957 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.216295 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.216299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.216301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.216584 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.216681 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.217576 (+ 895us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.217718 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260809 03:35:31.217943 13822 heartbeater.cc:347] Connected to a master server at 127.1.224.125:38527
I20260809 03:35:31.218032 13822 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:31.218223 13822 heartbeater.cc:511] Master 127.1.224.125:38527 requested a full tablet report, sending...
I20260809 03:35:31.218427 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.178591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.66:52069)
0809 03:35:31.179496 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:31.179509 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:31.183203 (+ 3694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:31.185950 (+ 2747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:31.185963 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:31.186121 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:31.186262 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.186271 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.187526 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.187530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.189942 (+ 2412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.189951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.190645 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.190649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.190748 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.208726 (+ 17978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:31.208742 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:31.212924 (+ 4182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:31.215480 (+ 2556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:31.215483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:31.215485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:31.215727 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:31.216076 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:31.216079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:31.216081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:31.216179 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:31.216746 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:31.216749 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:31.216864 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 03:35:31.218092 (+ 1228us) client_negotiation.cc:240] Negotiation successful
0809 03:35:31.218250 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":1627,"spinlock_wait_cycles":1280,"thread_start_us":153,"threads_started":1}
I20260809 03:35:31.219005 13024 ts_manager.cc:194] Registered new tserver with Master: eab5b9e49845493fb3bab869bc385008 (127.1.224.66:46675)
I20260809 03:35:31.219131 13821 heartbeater.cc:347] Connected to a master server at 127.1.224.126:39163
I20260809 03:35:31.219221 13821 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:31.219403 13821 heartbeater.cc:511] Master 127.1.224.126:39163 requested a full tablet report, sending...
I20260809 03:35:31.220142 12129 ts_manager.cc:194] Re-registered known tserver with Master: eab5b9e49845493fb3bab869bc385008 (127.1.224.66:46675)
I20260809 03:35:31.220834 12129 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:52069
I20260809 03:35:31.221036 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:41963
--local_ip_for_outbound_sockets=127.1.224.67
--tserver_master_addrs=127.1.224.126:39163
--webserver_port=46135
--webserver_interface=127.1.224.67
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab
--principal=oryx/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.224.126:39163,127.1.224.125:38527
--tserver_master_addrs=127.1.224.126:39163,127.1.224.125:38527,127.1.224.124:45879 with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:31.254529 13834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.250437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:45879 (local address 127.0.0.1:55234)
0809 03:35:31.250694 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:31.250708 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:31.250838 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:31.251169 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:31.251171 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:31.251187 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:31.251294 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.251298 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.252316 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.252318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.253183 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.253194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.254173 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.254176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.254249 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.254272 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:35:31.254327 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 03:35:31.254389 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":74,"threads_started":1}
I20260809 03:35:31.254567 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.250537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.1.224.124:45879)
0809 03:35:31.250683 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.250688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.250805 (+ 117us) server_negotiation.cc:407] Connection header received
0809 03:35:31.250955 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.250959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.251029 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.251148 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:31.251404 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.252200 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.253328 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.254046 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.254169 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.254333 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.254374 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.254439 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:35:31.255344 13349 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 03:35:31.255787 13391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 418b6f7249104662b195da37a5772666. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:31.255939 13391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: This master's current role is: LEARNER
W20260809 03:35:31.341032 13832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:31.341307 13832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:31.341351 13832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:31.345415 13832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:31.345471 13832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:31.345561 13832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:31.351156 13832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab
--principal=oryx/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:41963
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=46135
--tserver_master_addrs=127.1.224.126:39163,127.1.224.125:38527,127.1.224.124:45879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:31.352429 13832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:31.353592 13832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:31.364181 13841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:31.364180 13843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:31.364180 13839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:31.364374 13832 server_base.cc:1034] running on GCE node
I20260809 03:35:31.365041 13832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:31.365689 13832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:31.366861 13832 hybrid_clock.cc:648] HybridClock initialized: now 1786246531366843 us; error 30 us; skew 500 ppm
Aug 09 03:35:31 dist-test-slave-jq4t krb5kdc[12065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246531, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:31.369840 13832 init.cc:377] Logged in from keytab as oryx/127.1.224.67@KRBTEST.COM (short username oryx)
I20260809 03:35:31.370873 13832 webserver.cc:460] Webserver started at http://127.1.224.67:46135/ using document root <none> and password file <none>
I20260809 03:35:31.371484 13832 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:31.371596 13832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:31.375881 13832 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:31.378293 13856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:31.380023 13832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:31.380244 13832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "a4f7a00046b74bffb36c8796ffa67be1"
format_stamp: "Formatted at 2026-08-09 03:35:29 on dist-test-slave-jq4t"
server_key: "0a8cea7f5f53194206f590f8e016b414"
server_key_iv: "ffb7063cd017d7004e5dadbb9bfeed86"
server_key_version: "encryptionkey@0"
I20260809 03:35:31.381218 13832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:31.394960 13832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:31.398123 13832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:31.398295 13832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:31.398993 13832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:31.399904 13832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:31.399957 13832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:31.399999 13832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:31.400015 13832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:31.410060 13832 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:41963
I20260809 03:35:31.410100 14021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:41963 every 8 connection(s)
I20260809 03:35:31.411311 13832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:31.418985 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 13832
Aug 09 03:35:31 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246531, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
Aug 09 03:35:31 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246531, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.125@KRBTEST.COM
Aug 09 03:35:31 dist-test-slave-jq4t krb5kdc[12065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246531, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.124@KRBTEST.COM
I20260809 03:35:31.431152 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.413560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:55401 (local address 127.1.224.126:39163)
0809 03:35:31.413994 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.414000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.414684 (+ 684us) server_negotiation.cc:407] Connection header received
0809 03:35:31.415987 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.415994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.416060 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.416152 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.417707 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.418532 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.420434 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.420802 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.420893 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.425671 (+ 4778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.425676 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.425680 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.425713 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.428439 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.428987 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.428991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.428993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.429087 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.429894 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.429898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.429900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.430134 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.430203 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.430822 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.430968 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":331}
I20260809 03:35:31.431546 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.413700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:36871 (local address 127.1.224.124:45879)
0809 03:35:31.414123 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.414127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.415183 (+ 1056us) server_negotiation.cc:407] Connection header received
0809 03:35:31.416299 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.416302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.416379 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.416468 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.417707 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.418531 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.420462 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.420838 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.420924 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.425840 (+ 4916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.425846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.425849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.425878 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.427900 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.428626 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.428630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.428632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.428731 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.429563 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.429568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.429570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.429827 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.429930 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.431203 (+ 1273us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.431366 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342}
I20260809 03:35:31.432329 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.413945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.67:55401)
0809 03:35:31.414525 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:31.414579 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:31.415722 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:31.416324 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:31.416331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:31.416928 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:31.417537 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.417545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.418653 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.418656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.420270 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.420284 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.421140 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.421148 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.421376 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.422275 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:31.422298 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:31.425543 (+ 3245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:31.428603 (+ 3060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:31.428606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:31.428607 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:31.428862 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:31.429192 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:31.429195 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:31.429197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:31.429799 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:31.430278 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:31.430281 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:31.430447 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 03:35:31.430772 (+ 325us) client_negotiation.cc:240] Negotiation successful
0809 03:35:31.431175 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":284,"spinlock_wait_cycles":463616,"thread_start_us":176,"threads_started":1}
I20260809 03:35:31.433475 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.413945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:45879 (local address 127.1.224.67:36871)
0809 03:35:31.415067 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:31.415081 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:31.415722 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:31.416535 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:31.416538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:31.416935 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:31.417535 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.417545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.419054 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.419058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.420305 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.420316 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.421140 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.421144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.421764 (+ 620us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.423271 (+ 1507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:31.423285 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:31.425693 (+ 2408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:31.428049 (+ 2356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:31.428059 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:31.428083 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:31.428500 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:31.428852 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:31.428855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:31.428857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:31.429433 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:31.430016 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:31.430025 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:31.430445 (+ 420us) client_negotiation.cc:769] Sending connection context
0809 03:35:31.432103 (+ 1658us) client_negotiation.cc:240] Negotiation successful
0809 03:35:31.432253 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"mutex_wait_us":507,"spinlock_wait_cycles":36352,"thread_start_us":199,"threads_started":1}
I20260809 03:35:31.434397 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.413835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:57651 (local address 127.1.224.125:38527)
0809 03:35:31.414125 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.414131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.415434 (+ 1303us) server_negotiation.cc:407] Connection header received
0809 03:35:31.416041 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.416045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.416098 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.416154 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.417687 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.418249 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.420462 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.420838 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.420924 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.425275 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.425279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.425282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.425314 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.427653 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.432574 (+ 4921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.432579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.432583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.432690 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.433083 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.433091 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.433097 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.433366 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.433475 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.434134 (+ 659us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.434252 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260809 03:35:31.434964 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.413946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38527 (local address 127.1.224.67:57651)
0809 03:35:31.415314 (+ 1368us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:31.415329 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:31.415745 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:31.416336 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:31.416339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:31.416898 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:31.417524 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.417538 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.418406 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.418412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.420304 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.420316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.421140 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.421144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.421380 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.422468 (+ 1088us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:31.422482 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:31.425048 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:31.429316 (+ 4268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:31.429318 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:31.429320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:31.429657 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:31.432829 (+ 3172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:31.432834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:31.432837 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:31.432945 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:31.433819 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:31.433825 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:31.433927 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 03:35:31.434629 (+ 702us) client_negotiation.cc:240] Negotiation successful
0809 03:35:31.434778 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1286,"mutex_wait_us":680,"spinlock_wait_cycles":99072,"thread_start_us":106,"threads_started":1}
I20260809 03:35:31.435026 14022 heartbeater.cc:347] Connected to a master server at 127.1.224.124:45879
I20260809 03:35:31.435241 14042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.421762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56590 (local address 127.1.224.125:38527)
0809 03:35:31.422042 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.422045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.423447 (+ 1402us) server_negotiation.cc:407] Connection header received
0809 03:35:31.423542 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.423545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.423600 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.423697 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.425934 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.426763 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.428111 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.428466 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.428612 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.430460 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.430464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.430466 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.430493 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.433126 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.433703 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.433707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.433709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.433799 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.434150 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.434155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.434157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.434365 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.434546 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.434946 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.435075 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":79,"threads_started":1}
I20260809 03:35:31.435288 14022 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:31.435606 14026 heartbeater.cc:347] Connected to a master server at 127.1.224.125:38527
I20260809 03:35:31.435700 14026 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:31.435966 14026 heartbeater.cc:511] Master 127.1.224.125:38527 requested a full tablet report, sending...
I20260809 03:35:31.435966 14022 heartbeater.cc:511] Master 127.1.224.124:45879 requested a full tablet report, sending...
I20260809 03:35:31.437328 13034 ts_manager.cc:194] Registered new tserver with Master: a4f7a00046b74bffb36c8796ffa67be1 (127.1.224.67:41963)
I20260809 03:35:31.437640 13331 ts_manager.cc:194] Registered new tserver with Master: a4f7a00046b74bffb36c8796ffa67be1 (127.1.224.67:41963)
I20260809 03:35:31.438304 14023 heartbeater.cc:347] Connected to a master server at 127.1.224.126:39163
I20260809 03:35:31.438406 14023 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:31.438584 14023 heartbeater.cc:511] Master 127.1.224.126:39163 requested a full tablet report, sending...
I20260809 03:35:31.438670 14041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.421370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55120 (local address 127.1.224.126:39163)
0809 03:35:31.422700 (+ 1330us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.422716 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.423084 (+ 368us) server_negotiation.cc:407] Connection header received
0809 03:35:31.423136 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.423141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.425502 (+ 2361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.426292 (+ 790us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.427399 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.428448 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.429647 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.430015 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.430110 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.431800 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.431804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.431806 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.431832 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.433871 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.436330 (+ 2459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.436334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.436336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.436440 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.437597 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.437600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.437602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.437858 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.437997 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.438345 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.438495 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1235,"thread_start_us":118,"threads_started":1}
I20260809 03:35:31.439221 12130 ts_manager.cc:194] Re-registered known tserver with Master: a4f7a00046b74bffb36c8796ffa67be1 (127.1.224.67:41963)
I20260809 03:35:31.440012 12130 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:55401
I20260809 03:35:31.442065 12130 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 03:35:31.445081 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.422230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55246 (local address 127.1.224.124:45879)
0809 03:35:31.423739 (+ 1509us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.423744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.423757 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:31.423795 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.423798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.423843 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.424083 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.424899 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.437348 (+ 12449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.439241 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.439639 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.439752 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.441601 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.441604 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.441606 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.441630 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.443171 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.443663 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.443665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.443666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.443746 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.444133 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.444135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.444136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.444325 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.444447 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.444746 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.444866 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1425,"thread_start_us":116,"threads_started":1}
W20260809 03:35:31.446328 12130 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:31.449405 13833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: NON_VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: true } } }
I20260809 03:35:31.449569 13833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:31.453327 13833 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666: attempting to promote NON_VOTER 8ff6a3b7018b4fac9abff079bdbae592 to VOTER
I20260809 03:35:31.453715 13833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } }
I20260809 03:35:31.455474 13046 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 FOLLOWER]: Refusing update from remote peer 418b6f7249104662b195da37a5772666: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 03:35:31.455498 13349 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 1 LEARNER]: Refusing update from remote peer 418b6f7249104662b195da37a5772666: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 03:35:31.455937 14049 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 03:35:31.456094 14051 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 03:35:31.457849 14051 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 8ff6a3b7018b4fac9abff079bdbae592 (127.1.224.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } } }
I20260809 03:35:31.458252 13046 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 8ff6a3b7018b4fac9abff079bdbae592 (127.1.224.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } } }
I20260809 03:35:31.460098 14051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } } }
I20260809 03:35:31.460223 14051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:31.460429 14051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } } }
I20260809 03:35:31.460585 14051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:31.461159 14063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } } }
I20260809 03:35:31.461359 14063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:35:31.461612 13833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } } }
I20260809 03:35:31.461716 13833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:31.462560 13349 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 8ff6a3b7018b4fac9abff079bdbae592 (127.1.224.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } } }
I20260809 03:35:31.467092 13391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "418b6f7249104662b195da37a5772666" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } } }
I20260809 03:35:31.467224 13391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:35:31.471081 14068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.463648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:46675 (local address 127.0.0.1:43404)
0809 03:35:31.464420 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:31.464436 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:31.465024 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:31.465984 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:31.465997 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:31.466033 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:31.466375 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.466382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.468040 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.468044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.469588 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.469598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.470598 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.470602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.470706 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.470752 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 03:35:31.470823 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 03:35:31.470925 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":3,"spinlock_wait_cycles":4736}
I20260809 03:35:31.471168 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.463820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43404 (local address 127.1.224.66:46675)
0809 03:35:31.464357 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.464372 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.464986 (+ 614us) server_negotiation.cc:407] Connection header received
0809 03:35:31.465586 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.465592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.465814 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.466520 (+ 706us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:31.466662 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.467866 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.469693 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.470456 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.470578 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.470738 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.470891 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.471016 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":138,"threads_started":1}
I20260809 03:35:31.471309 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.464738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40244 (local address 127.1.224.65:33961)
0809 03:35:31.465439 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.465446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.465471 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:35:31.465559 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.465565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.465811 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.466121 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:31.466279 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.467423 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.469206 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.470678 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.470849 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.471000 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.471055 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.471166 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":97,"threads_started":1}
I20260809 03:35:31.471238 13834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.464153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33961 (local address 127.0.0.1:40244)
0809 03:35:31.464727 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:31.464741 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:31.464847 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:31.465989 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:31.465992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:31.466031 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:31.466172 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.466179 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.467566 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.467569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.469071 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.469082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.470852 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.470856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.470946 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.470967 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:35:31.471030 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 03:35:31.471102 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":248,"spinlock_wait_cycles":10880,"thread_start_us":106,"threads_started":1}
I20260809 03:35:31.472107 14071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.465055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:41963 (local address 127.0.0.1:59234)
0809 03:35:31.465291 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:31.465306 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:31.465397 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:31.466486 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:31.466500 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:31.466518 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:31.466596 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.466601 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.467992 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.467996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.469498 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.469506 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.471674 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.471680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.471777 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.471800 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:35:31.471906 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 03:35:31.471980 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":23,"thread_start_us":101,"threads_started":1}
I20260809 03:35:31.472380 14069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.464664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59234 (local address 127.1.224.67:41963)
0809 03:35:31.464970 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.464983 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.465929 (+ 946us) server_negotiation.cc:407] Connection header received
0809 03:35:31.466016 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.466021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.466190 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.466336 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:31.466729 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.467851 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.469617 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.470456 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.470578 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.470738 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.471945 (+ 1207us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.472055 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":116,"threads_started":1}
I20260809 03:35:31.474910 13505 tablet_service.cc:1467] Processing CreateTablet for tablet e931f4c3a0f94087ac531db468a13b7d (DEFAULT_TABLE table=test-table [id=ba6db57449de43ad817214cade7a93b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:31.474921 13720 tablet_service.cc:1467] Processing CreateTablet for tablet e931f4c3a0f94087ac531db468a13b7d (DEFAULT_TABLE table=test-table [id=ba6db57449de43ad817214cade7a93b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:31.476159 13720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e931f4c3a0f94087ac531db468a13b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:31.476159 13505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e931f4c3a0f94087ac531db468a13b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:31.479140 13931 tablet_service.cc:1467] Processing CreateTablet for tablet e931f4c3a0f94087ac531db468a13b7d (DEFAULT_TABLE table=test-table [id=ba6db57449de43ad817214cade7a93b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:31.480237 13931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e931f4c3a0f94087ac531db468a13b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:31.485014 14072 tablet_bootstrap.cc:492] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0: Bootstrap starting.
I20260809 03:35:31.485289 14073 tablet_bootstrap.cc:492] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008: Bootstrap starting.
I20260809 03:35:31.488112 14074 tablet_bootstrap.cc:492] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1: Bootstrap starting.
I20260809 03:35:31.489851 14073 tablet_bootstrap.cc:654] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:31.490191 14074 tablet_bootstrap.cc:654] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:31.490667 14072 tablet_bootstrap.cc:654] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:31.491101 14074 log.cc:824] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:31.491451 14072 log.cc:824] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:31.492205 14073 log.cc:824] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:31.493350 14074 tablet_bootstrap.cc:492] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1: No bootstrap required, opened a new log
I20260809 03:35:31.493568 14074 ts_tablet_manager.cc:1397] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 03:35:31.494024 14072 tablet_bootstrap.cc:492] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0: No bootstrap required, opened a new log
I20260809 03:35:31.494228 14072 ts_tablet_manager.cc:1397] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260809 03:35:31.494256 14073 tablet_bootstrap.cc:492] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008: No bootstrap required, opened a new log
I20260809 03:35:31.494421 14073 ts_tablet_manager.cc:1397] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.001s
I20260809 03:35:31.497256 14072 raft_consensus.cc:348] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } } peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } }
I20260809 03:35:31.497543 14072 raft_consensus.cc:374] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:31.497601 14072 raft_consensus.cc:729] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca54a7b094841a5883552c27c0e94c0, State: Initialized, Role: FOLLOWER
I20260809 03:35:31.497684 14074 raft_consensus.cc:348] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } } peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } }
I20260809 03:35:31.497994 14074 raft_consensus.cc:374] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:31.498065 14074 raft_consensus.cc:729] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f7a00046b74bffb36c8796ffa67be1, State: Initialized, Role: FOLLOWER
I20260809 03:35:31.498157 14072 consensus_queue.cc:260] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } } peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } }
I20260809 03:35:31.498559 14074 consensus_queue.cc:260] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } } peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } }
I20260809 03:35:31.499015 14073 raft_consensus.cc:348] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } } peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } }
I20260809 03:35:31.499250 14073 raft_consensus.cc:374] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:31.499270 14072 ts_tablet_manager.cc:1428] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 03:35:31.499297 14073 raft_consensus.cc:729] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab5b9e49845493fb3bab869bc385008, State: Initialized, Role: FOLLOWER
I20260809 03:35:31.499552 14074 ts_tablet_manager.cc:1428] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:35:31.500643 13610 heartbeater.cc:503] Master 127.1.224.126:39163 was elected leader, sending a full tablet report...
I20260809 03:35:31.500638 14023 heartbeater.cc:503] Master 127.1.224.126:39163 was elected leader, sending a full tablet report...
I20260809 03:35:31.499732 14073 consensus_queue.cc:260] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } } peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } }
I20260809 03:35:31.502476 14073 ts_tablet_manager.cc:1428] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008: Time spent starting tablet: real 0.008s user 0.005s sys 0.002s
I20260809 03:35:31.503460 13821 heartbeater.cc:503] Master 127.1.224.126:39163 was elected leader, sending a full tablet report...
W20260809 03:35:31.662967 14027 tablet.cc:2369] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:35:31.675024 13612 tablet.cc:2369] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:35:31.678143 13825 tablet.cc:2369] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:31.759610 14080 raft_consensus.cc:482] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:31.759871 14080 raft_consensus.cc:504] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } } peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } }
I20260809 03:35:31.761229 14080 leader_election.cc:288] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eab5b9e49845493fb3bab869bc385008 (127.1.224.66:46675), bca54a7b094841a5883552c27c0e94c0 (127.1.224.65:33961)
I20260809 03:35:31.766386 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.761853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:51835 (local address 127.1.224.66:46675)
0809 03:35:31.762009 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.762013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.762027 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:35:31.762077 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.762080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.762130 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.762207 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:31.762618 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.763417 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.765316 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.765739 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.765880 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.765944 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.766151 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.766238 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 03:35:31.766386 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.761489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:46675 (local address 127.1.224.67:51835)
0809 03:35:31.761605 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:31.761624 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:31.761731 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:31.762239 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:31.762242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:31.762276 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:31.762490 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.762497 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.763537 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.763541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.765150 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.765163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.765881 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.765885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.765998 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.766022 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:35:31.766127 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 03:35:31.766237 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":220,"spinlock_wait_cycles":9728}
I20260809 03:35:31.766527 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.761498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33961 (local address 127.1.224.67:50053)
0809 03:35:31.761759 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:31.761774 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:31.761889 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:31.762239 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:31.762242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:31.762275 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:31.762489 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.762497 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.763537 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.763541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.765150 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:31.765163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.766108 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.766112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:31.766219 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.766243 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:35:31.766328 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 03:35:31.766404 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":357,"spinlock_wait_cycles":4736}
I20260809 03:35:31.767052 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.761613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:50053 (local address 127.1.224.65:33961)
0809 03:35:31.761754 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.761759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.761880 (+ 121us) server_negotiation.cc:407] Connection header received
0809 03:35:31.762028 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.762032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.762091 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.762178 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:31.762618 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.763422 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.765301 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.765854 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.765974 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.766037 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.766836 (+ 799us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.766924 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:35:31.767428 13749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e931f4c3a0f94087ac531db468a13b7d" candidate_uuid: "a4f7a00046b74bffb36c8796ffa67be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eab5b9e49845493fb3bab869bc385008" is_pre_election: true
I20260809 03:35:31.767585 13537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e931f4c3a0f94087ac531db468a13b7d" candidate_uuid: "a4f7a00046b74bffb36c8796ffa67be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca54a7b094841a5883552c27c0e94c0" is_pre_election: true
I20260809 03:35:31.767822 13749 raft_consensus.cc:2454] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4f7a00046b74bffb36c8796ffa67be1 in term 0.
I20260809 03:35:31.767861 13537 raft_consensus.cc:2454] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4f7a00046b74bffb36c8796ffa67be1 in term 0.
I20260809 03:35:31.768378 13857 leader_election.cc:302] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4f7a00046b74bffb36c8796ffa67be1, eab5b9e49845493fb3bab869bc385008; no voters:
I20260809 03:35:31.768723 14080 raft_consensus.cc:2781] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:31.768877 14080 raft_consensus.cc:482] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:31.768959 14080 raft_consensus.cc:3037] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:31.770061 14080 raft_consensus.cc:504] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } } peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } }
I20260809 03:35:31.770481 14080 leader_election.cc:288] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [CANDIDATE]: Term 1 election: Requested vote from peers eab5b9e49845493fb3bab869bc385008 (127.1.224.66:46675), bca54a7b094841a5883552c27c0e94c0 (127.1.224.65:33961)
I20260809 03:35:31.770880 13537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e931f4c3a0f94087ac531db468a13b7d" candidate_uuid: "a4f7a00046b74bffb36c8796ffa67be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca54a7b094841a5883552c27c0e94c0"
I20260809 03:35:31.770889 13749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e931f4c3a0f94087ac531db468a13b7d" candidate_uuid: "a4f7a00046b74bffb36c8796ffa67be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eab5b9e49845493fb3bab869bc385008"
I20260809 03:35:31.771035 13749 raft_consensus.cc:3037] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:31.771041 13537 raft_consensus.cc:3037] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:31.772197 13537 raft_consensus.cc:2454] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4f7a00046b74bffb36c8796ffa67be1 in term 1.
I20260809 03:35:31.772276 13749 raft_consensus.cc:2454] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4f7a00046b74bffb36c8796ffa67be1 in term 1.
I20260809 03:35:31.772607 13857 leader_election.cc:302] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4f7a00046b74bffb36c8796ffa67be1, eab5b9e49845493fb3bab869bc385008; no voters:
I20260809 03:35:31.772843 14080 raft_consensus.cc:2781] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:31.773082 14080 raft_consensus.cc:686] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 1 LEADER]: Becoming Leader. State: Replica: a4f7a00046b74bffb36c8796ffa67be1, State: Running, Role: LEADER
I20260809 03:35:31.773484 14080 consensus_queue.cc:237] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } } peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } }
I20260809 03:35:31.778538 12130 catalog_manager.cc:5495] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4f7a00046b74bffb36c8796ffa67be1 (127.1.224.67). New cstate: current_term: 1 leader_uuid: "a4f7a00046b74bffb36c8796ffa67be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } health_report { overall_health: HEALTHY } } }
I20260809 03:35:31.833352 14069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.829488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59238 (local address 127.1.224.67:41963)
0809 03:35:31.829622 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.829628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.829645 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:35:31.829757 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.829760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.829818 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.829939 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:31.830214 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.830770 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.831717 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.831950 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.832069 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.832404 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:31.832930 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:31.833075 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.833116 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.833177 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:35:31.839641 13537 raft_consensus.cc:1261] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0 [term 1 FOLLOWER]: Refusing update from remote peer a4f7a00046b74bffb36c8796ffa67be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:31.839641 13749 raft_consensus.cc:1261] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008 [term 1 FOLLOWER]: Refusing update from remote peer a4f7a00046b74bffb36c8796ffa67be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:31.840456 14080 consensus_queue.cc:1035] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:31.840744 14085 consensus_queue.cc:1035] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:31.859380 12130 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:31.859545 12130 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:31.864477 12130 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 418b6f7249104662b195da37a5772666: Sending DeleteTablet for 3 replicas of tablet e931f4c3a0f94087ac531db468a13b7d
I20260809 03:35:31.865284 13720 tablet_service.cc:1514] Processing DeleteTablet for tablet e931f4c3a0f94087ac531db468a13b7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:31 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:43404
I20260809 03:35:31.865628 13505 tablet_service.cc:1514] Processing DeleteTablet for tablet e931f4c3a0f94087ac531db468a13b7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:31 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:40244
I20260809 03:35:31.865972 14103 tablet_replica.cc:330] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008: stopping tablet replica
I20260809 03:35:31.866333 13931 tablet_service.cc:1514] Processing DeleteTablet for tablet e931f4c3a0f94087ac531db468a13b7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:31 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:59234
I20260809 03:35:31.866430 14103 raft_consensus.cc:2229] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:31.866928 14103 raft_consensus.cc:2258] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:31.868309 14104 tablet_replica.cc:330] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0: stopping tablet replica
I20260809 03:35:31.868675 14104 raft_consensus.cc:2229] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:31.868790 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 12086
I20260809 03:35:31.875906 14103 ts_tablet_manager.cc:1905] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:31.879284 14105 tablet_replica.cc:330] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1: stopping tablet replica
I20260809 03:35:31.879487 14103 ts_tablet_manager.cc:1918] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:31.879666 14103 log.cc:1196] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/e931f4c3a0f94087ac531db468a13b7d
I20260809 03:35:31.879750 14105 raft_consensus.cc:2229] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:31.880272 14103 ts_tablet_manager.cc:1939] T e931f4c3a0f94087ac531db468a13b7d P eab5b9e49845493fb3bab869bc385008: Deleting consensus metadata
I20260809 03:35:31.880529 14104 raft_consensus.cc:2258] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:31.880834 14105 raft_consensus.cc:2258] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:31.882692 14104 ts_tablet_manager.cc:1905] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:31.885407 14104 ts_tablet_manager.cc:1918] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:31.885560 14104 log.cc:1196] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-0/wal/wals/e931f4c3a0f94087ac531db468a13b7d
I20260809 03:35:31.886806 14105 ts_tablet_manager.cc:1905] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:31.889787 14105 ts_tablet_manager.cc:1918] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:31.889914 14105 log.cc:1196] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/e931f4c3a0f94087ac531db468a13b7d
I20260809 03:35:31.890229 14105 ts_tablet_manager.cc:1939] T e931f4c3a0f94087ac531db468a13b7d P a4f7a00046b74bffb36c8796ffa67be1: Deleting consensus metadata
W20260809 03:35:31.891458 13647 connection.cc:535] server connection from 127.0.0.1:43404 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:35:31.891856 14104 ts_tablet_manager.cc:1939] T e931f4c3a0f94087ac531db468a13b7d P bca54a7b094841a5883552c27c0e94c0: Deleting consensus metadata
W20260809 03:35:31.892133 13860 connection.cc:441] server connection from 127.0.0.1:59234 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59234 (request call id 1) could send its response
W20260809 03:35:31.893124 13436 connection.cc:441] server connection from 127.0.0.1:40244 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 1) could send its response
I20260809 03:35:31.903851 14042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.867870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56602 (local address 127.1.224.125:38527)
0809 03:35:31.867986 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.867990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.893083 (+ 25093us) server_negotiation.cc:407] Connection header received
0809 03:35:31.893226 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.893229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.893295 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.893384 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.894489 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.895061 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.896914 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.897317 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.897419 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.899217 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.899221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.899224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.899253 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.901336 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.901849 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.901856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.901858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.901962 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.902330 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.902333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.902335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.902614 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.902770 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.903450 (+ 680us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.903621 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:35:31.907291 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.896254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56616 (local address 127.1.224.125:38527)
0809 03:35:31.896403 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.896408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.896422 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:35:31.896465 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.896468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.896538 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.896612 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.897582 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.898363 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.899556 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.899891 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.899959 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.901688 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.901692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.901695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.901720 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.904219 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.905977 (+ 1758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.905981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.905982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.906072 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.906374 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.906377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.906379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.906583 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.906689 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.907024 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.907138 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:35:31.907398 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.896836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55262 (local address 127.1.224.124:45879)
0809 03:35:31.897628 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.897635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.897653 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:35:31.897706 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.897711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.897773 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.897854 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.898887 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.899674 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.901050 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.901417 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.901493 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.903214 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.903218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.903220 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.903251 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.905339 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.906031 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.906035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.906037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.906130 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.906431 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.906434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.906436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.906667 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.906774 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.907122 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.907240 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683}
I20260809 03:35:31.917340 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:31.867516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55260 (local address 127.1.224.124:45879)
0809 03:35:31.867633 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:31.867638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:31.904596 (+ 36958us) server_negotiation.cc:407] Connection header received
0809 03:35:31.904794 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:31.904797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:31.904857 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:31.904912 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:31.905767 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.906550 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.908016 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:31.908373 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:31.908494 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:31.909921 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:31.909925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:31.909928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:31.909953 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:31.915456 (+ 5503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.915923 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.915927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.915928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.916016 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:31.916322 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:31.916325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:31.916327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:31.916606 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:31.916749 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:31.917023 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 03:35:31.917156 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:35:32.853837 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:32.853296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.66:41469)
0809 03:35:32.853604 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:32.853714 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":98,"threads_started":1}
W20260809 03:35:32.854177 13821 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:39163 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
I20260809 03:35:32.855445 14115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:32.855012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.67:36771)
0809 03:35:32.855235 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:32.855348 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":87,"threads_started":1}
W20260809 03:35:32.855996 14023 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:39163 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
I20260809 03:35:32.856542 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:32.855913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.65:42575)
0809 03:35:32.856320 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:32.856419 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":83,"threads_started":1}
W20260809 03:35:32.856858 13610 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:39163 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
I20260809 03:35:33.364229 14117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 418b6f7249104662b195da37a5772666)
I20260809 03:35:33.364467 14117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } }
I20260809 03:35:33.365638 14117 leader_election.cc:288] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 418b6f7249104662b195da37a5772666 (127.1.224.126:39163), 8ff6a3b7018b4fac9abff079bdbae592 (127.1.224.124:45879)
I20260809 03:35:33.366524 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.365898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55600)
0809 03:35:33.366294 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.366401 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":82,"threads_started":1}
W20260809 03:35:33.366744 13014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
I20260809 03:35:33.367666 14123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 418b6f7249104662b195da37a5772666)
I20260809 03:35:33.367820 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.367538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55608)
0809 03:35:33.367667 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.367721 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 03:35:33.367871 14123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } }
W20260809 03:35:33.367981 13014 leader_election.cc:334] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 418b6f7249104662b195da37a5772666 (127.1.224.126:39163): Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
I20260809 03:35:33.369092 14123 leader_election.cc:288] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 418b6f7249104662b195da37a5772666 (127.1.224.126:39163), 9abcb73ba3c448b79a20e82f3bf2b131 (127.1.224.125:38527)
I20260809 03:35:33.369969 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.369291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55616)
0809 03:35:33.369796 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.369884 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":55,"thread_start_us":85,"threads_started":1}
W20260809 03:35:33.370571 13307 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
I20260809 03:35:33.371069 14119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.365898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:45879 (local address 127.0.0.1:55278)
0809 03:35:33.366324 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.366339 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:33.366476 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:33.366864 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:33.366868 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:33.366892 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:33.367032 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.367044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.368190 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.368194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.369483 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.369496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.370629 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.370632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.370721 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.370745 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:35:33.370818 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 03:35:33.370924 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":30,"thread_start_us":57,"threads_started":1}
I20260809 03:35:33.371658 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.371407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55632)
0809 03:35:33.371540 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.371589 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 03:35:33.371694 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.366134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55278 (local address 127.1.224.124:45879)
0809 03:35:33.366352 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:33.366356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:33.366404 (+ 48us) server_negotiation.cc:407] Connection header received
0809 03:35:33.366646 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:33.366652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:33.366712 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:33.366824 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:33.367184 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.368038 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.369663 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.370379 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.370499 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.370568 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:33.371327 (+ 759us) server_negotiation.cc:299] Negotiation successful
0809 03:35:33.371568 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":154,"thread_start_us":64,"threads_started":1}
W20260809 03:35:33.371856 13307 leader_election.cc:334] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 418b6f7249104662b195da37a5772666 (127.1.224.126:39163): Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
I20260809 03:35:33.372421 13349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8ff6a3b7018b4fac9abff079bdbae592" is_pre_election: true
I20260809 03:35:33.372680 13349 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9abcb73ba3c448b79a20e82f3bf2b131 in term 1.
I20260809 03:35:33.373032 13016 leader_election.cc:302] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ff6a3b7018b4fac9abff079bdbae592, 9abcb73ba3c448b79a20e82f3bf2b131; no voters: 418b6f7249104662b195da37a5772666
I20260809 03:35:33.373332 14117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 03:35:33.373413 14117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 418b6f7249104662b195da37a5772666)
I20260809 03:35:33.373464 14117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:35:33.374006 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.369291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38527 (local address 127.0.0.1:56626)
0809 03:35:33.369641 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.369658 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:33.369758 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:33.370065 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:33.370070 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:33.370096 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:33.370196 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.370202 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.371211 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.371214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.372429 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.372439 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.373561 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.373565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.373681 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.373718 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 03:35:33.373805 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 03:35:33.373880 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":80,"threads_started":1}
I20260809 03:35:33.374037 14127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.369488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56626 (local address 127.1.224.125:38527)
0809 03:35:33.369818 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:33.369828 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:33.369842 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:35:33.369884 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:33.369887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:33.369934 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:33.370024 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:33.370331 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.371075 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.372664 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.373284 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.373397 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.373477 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:33.373821 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 03:35:33.373920 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":122,"threads_started":1}
I20260809 03:35:33.374704 13046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8ff6a3b7018b4fac9abff079bdbae592" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" is_pre_election: true
I20260809 03:35:33.375020 14117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } }
I20260809 03:35:33.375308 13046 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ff6a3b7018b4fac9abff079bdbae592 in current term 2: Already voted for candidate 9abcb73ba3c448b79a20e82f3bf2b131 in this term.
I20260809 03:35:33.375492 14117 leader_election.cc:288] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [CANDIDATE]: Term 2 election: Requested vote from peers 418b6f7249104662b195da37a5772666 (127.1.224.126:39163), 8ff6a3b7018b4fac9abff079bdbae592 (127.1.224.124:45879)
I20260809 03:35:33.376078 13349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8ff6a3b7018b4fac9abff079bdbae592"
I20260809 03:35:33.376207 13349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:35:33.376688 13308 leader_election.cc:302] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ff6a3b7018b4fac9abff079bdbae592; no voters: 418b6f7249104662b195da37a5772666, 9abcb73ba3c448b79a20e82f3bf2b131
I20260809 03:35:33.377467 13349 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9abcb73ba3c448b79a20e82f3bf2b131 in term 2.
I20260809 03:35:33.377655 14123 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:35:33.377791 13016 leader_election.cc:302] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ff6a3b7018b4fac9abff079bdbae592, 9abcb73ba3c448b79a20e82f3bf2b131; no voters:
I20260809 03:35:33.378029 14117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:35:33.378315 14117 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [term 2 LEADER]: Becoming Leader. State: Replica: 9abcb73ba3c448b79a20e82f3bf2b131, State: Running, Role: LEADER
I20260809 03:35:33.378594 14117 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } }
I20260809 03:35:33.379760 14131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9abcb73ba3c448b79a20e82f3bf2b131. Latest consensus state: current_term: 2 leader_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } } }
I20260809 03:35:33.379926 14119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.375622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55638)
0809 03:35:33.379734 (+ 4112us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.379843 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4018}
I20260809 03:35:33.380038 14131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:33.380632 14133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:33.380805 14119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.380543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55654)
0809 03:35:33.380666 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.380733 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 03:35:33.380962 13014 leader_election.cc:334] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 418b6f7249104662b195da37a5772666 (127.1.224.126:39163): Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
I20260809 03:35:33.384570 14133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:33.385053 14133 catalog_manager.cc:1259] Loaded cluster ID: 83dc0a9e3c9e46ad9d196726bcc3072b
I20260809 03:35:33.385138 14133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:33.385975 14133 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:33.386488 14133 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: Loaded TSK: 0
I20260809 03:35:33.387650 14133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:35:33.434542 13024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:33.437886 13024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:33.441341 14119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.441035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55674)
0809 03:35:33.441212 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.441266 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 03:35:33.442268 14119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.441949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55682)
0809 03:35:33.442087 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.442186 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 03:35:33.442305 13349 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [term 2 FOLLOWER]: Refusing update from remote peer 9abcb73ba3c448b79a20e82f3bf2b131: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
W20260809 03:35:33.442402 13014 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 -> Peer 418b6f7249104662b195da37a5772666 (127.1.224.126:39163): Couldn't send request to peer 418b6f7249104662b195da37a5772666. Status: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 03:35:33.443015 14131 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 03:35:33.445369 14123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9abcb73ba3c448b79a20e82f3bf2b131. Latest consensus state: current_term: 2 leader_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } } }
I20260809 03:35:33.445566 14123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:35:33.446743 14117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } } }
I20260809 03:35:33.446889 14117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:33.447602 14123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "418b6f7249104662b195da37a5772666" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39163 } } peers { permanent_uuid: "9abcb73ba3c448b79a20e82f3bf2b131" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38527 } attrs { promote: false } } peers { permanent_uuid: "8ff6a3b7018b4fac9abff079bdbae592" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 45879 } attrs { promote: false } } }
I20260809 03:35:33.447732 14123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ff6a3b7018b4fac9abff079bdbae592 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:35:33.448218 14136 mvcc.cc:205] Tried to move back new op lower bound from 7316465800969928704 to 7316465800721461248. Current Snapshot: MvccSnapshot[applied={T|T < 7316465800969928704 or (T in {7316465800969928704})}]
I20260809 03:35:33.458572 14119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.453292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33961 (local address 127.0.0.1:40258)
0809 03:35:33.453405 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.453424 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:33.453522 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:33.454405 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:33.454408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:33.454432 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:33.454549 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.454555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.455589 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.455593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.456684 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.456694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.458021 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.458025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.458109 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.458152 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 03:35:33.458203 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 03:35:33.458283 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 03:35:33.458820 14145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.454016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43406 (local address 127.1.224.66:46675)
0809 03:35:33.454322 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:33.454328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:33.455037 (+ 709us) server_negotiation.cc:407] Connection header received
0809 03:35:33.455127 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:33.455130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:33.455182 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:33.455278 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:33.455528 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.456301 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.457715 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.458263 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.458478 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.458568 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:33.458609 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 03:35:33.458680 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":113,"threads_started":1}
I20260809 03:35:33.458817 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.453560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:41963 (local address 127.0.0.1:59242)
0809 03:35:33.453662 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.453676 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:33.453767 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:33.454741 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:33.454744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:33.454763 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:33.454868 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.454873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.456094 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.456098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.457267 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.457276 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.458214 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.458219 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.458324 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.458357 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 03:35:33.458584 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 03:35:33.458671 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":56}
I20260809 03:35:33.458813 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.453672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:46675 (local address 127.0.0.1:43406)
0809 03:35:33.454928 (+ 1256us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.454942 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:33.455043 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:33.455306 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:33.455309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:33.455326 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:33.455409 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.455414 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.456417 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.456421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.457605 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.457615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.458377 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.458380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.458465 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.458494 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 03:35:33.458555 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 03:35:33.458639 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1193,"mutex_wait_us":2,"thread_start_us":116,"threads_started":1}
I20260809 03:35:33.459344 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.453828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.1.224.65:33961)
0809 03:35:33.454158 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:33.454164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:33.454181 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:35:33.454240 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:33.454244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:33.454294 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:33.454388 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:33.454688 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.455496 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.456829 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.457879 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.458884 (+ 1005us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.459084 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:33.459129 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 03:35:33.459211 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":98,"threads_started":1}
I20260809 03:35:33.459530 13720 tablet_service.cc:1467] Processing CreateTablet for tablet 329e55b0de274cd6abebbe76dd17f835 (DEFAULT_TABLE table=test-table [id=f7e2c079841f4123b8bc637103f3abd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:33.459762 14146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.454189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59242 (local address 127.1.224.67:41963)
0809 03:35:33.454480 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:33.454484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:33.454497 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:33.454548 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:33.454551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:33.454611 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:33.454963 (+ 352us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:33.455115 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.455986 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.457451 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.458091 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.458203 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.458269 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:33.459506 (+ 1237us) server_negotiation.cc:299] Negotiation successful
0809 03:35:33.459610 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":78,"threads_started":1}
I20260809 03:35:33.459894 13720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 329e55b0de274cd6abebbe76dd17f835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:33.459908 13505 tablet_service.cc:1467] Processing CreateTablet for tablet 329e55b0de274cd6abebbe76dd17f835 (DEFAULT_TABLE table=test-table [id=f7e2c079841f4123b8bc637103f3abd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:33.460376 13505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 329e55b0de274cd6abebbe76dd17f835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:33.460373 13931 tablet_service.cc:1467] Processing CreateTablet for tablet 329e55b0de274cd6abebbe76dd17f835 (DEFAULT_TABLE table=test-table [id=f7e2c079841f4123b8bc637103f3abd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:33.460688 13931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 329e55b0de274cd6abebbe76dd17f835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:33.462626 14148 tablet_bootstrap.cc:492] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008: Bootstrap starting.
I20260809 03:35:33.463366 14149 tablet_bootstrap.cc:492] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0: Bootstrap starting.
I20260809 03:35:33.463790 14151 tablet_bootstrap.cc:492] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1: Bootstrap starting.
I20260809 03:35:33.463812 14148 tablet_bootstrap.cc:654] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:33.464359 14149 tablet_bootstrap.cc:654] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:33.464727 14151 tablet_bootstrap.cc:654] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:33.465288 14148 tablet_bootstrap.cc:492] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008: No bootstrap required, opened a new log
I20260809 03:35:33.465411 14148 ts_tablet_manager.cc:1397] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:33.465787 14149 tablet_bootstrap.cc:492] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0: No bootstrap required, opened a new log
I20260809 03:35:33.465883 14149 ts_tablet_manager.cc:1397] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:33.465937 14148 raft_consensus.cc:348] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } } peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } }
I20260809 03:35:33.466076 14148 raft_consensus.cc:374] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:33.466111 14148 raft_consensus.cc:729] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab5b9e49845493fb3bab869bc385008, State: Initialized, Role: FOLLOWER
I20260809 03:35:33.466259 14148 consensus_queue.cc:260] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } } peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } }
I20260809 03:35:33.466382 14149 raft_consensus.cc:348] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } } peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } }
I20260809 03:35:33.466509 14149 raft_consensus.cc:374] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:33.466553 14149 raft_consensus.cc:729] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca54a7b094841a5883552c27c0e94c0, State: Initialized, Role: FOLLOWER
I20260809 03:35:33.466639 14148 ts_tablet_manager.cc:1428] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 03:35:33.466718 14149 consensus_queue.cc:260] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } } peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } }
I20260809 03:35:33.467051 14149 ts_tablet_manager.cc:1428] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 03:35:33.469139 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.467839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.65:39413)
0809 03:35:33.468945 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.469035 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1027,"thread_start_us":96,"threads_started":1}
I20260809 03:35:33.469738 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.466992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.66:44737)
0809 03:35:33.469568 (+ 2576us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.469629 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2507,"thread_start_us":115,"threads_started":1}
I20260809 03:35:33.470728 13822 heartbeater.cc:503] Master 127.1.224.125:38527 was elected leader, sending a full tablet report...
I20260809 03:35:33.470880 14161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.470200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.66:39063)
0809 03:35:33.470704 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.470758 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":107,"threads_started":1}
I20260809 03:35:33.470922 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.470660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.65:38087)
0809 03:35:33.470797 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.470860 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 03:35:33.471186 13611 heartbeater.cc:503] Master 127.1.224.125:38527 was elected leader, sending a full tablet report...
I20260809 03:35:33.471235 14151 tablet_bootstrap.cc:492] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1: No bootstrap required, opened a new log
I20260809 03:35:33.471411 14151 ts_tablet_manager.cc:1397] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260809 03:35:33.471941 14151 raft_consensus.cc:348] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } } peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } }
I20260809 03:35:33.472086 14151 raft_consensus.cc:374] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:33.472136 14151 raft_consensus.cc:729] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f7a00046b74bffb36c8796ffa67be1, State: Initialized, Role: FOLLOWER
I20260809 03:35:33.472293 14151 consensus_queue.cc:260] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } } peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } }
I20260809 03:35:33.472757 14151 ts_tablet_manager.cc:1428] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:33.473515 14163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.473122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.67:33977)
0809 03:35:33.473373 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.473436 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":95,"threads_started":1}
I20260809 03:35:33.474210 14026 heartbeater.cc:503] Master 127.1.224.125:38527 was elected leader, sending a full tablet report...
I20260809 03:35:33.474298 14163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.473984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.67:53769)
0809 03:35:33.474140 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.474193 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 03:35:33.928958 14156 raft_consensus.cc:482] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:33.929153 14156 raft_consensus.cc:504] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } } peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } }
I20260809 03:35:33.930343 14156 leader_election.cc:288] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca54a7b094841a5883552c27c0e94c0 (127.1.224.65:33961), a4f7a00046b74bffb36c8796ffa67be1 (127.1.224.67:41963)
I20260809 03:35:33.935783 14146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.931095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:57165 (local address 127.1.224.67:41963)
0809 03:35:33.931240 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:33.931247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:33.931262 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:35:33.931322 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:33.931324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:33.931362 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:33.931467 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:33.931753 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.932596 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.934414 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.935032 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.935177 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.935234 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:33.935565 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 03:35:33.935642 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:35:33.935783 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.930699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:34577 (local address 127.1.224.65:33961)
0809 03:35:33.930876 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:33.930881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:33.931123 (+ 242us) server_negotiation.cc:407] Connection header received
0809 03:35:33.931725 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:33.931729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:33.931780 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:33.931868 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:33.932100 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.932804 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.934414 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.935032 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.935177 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.935234 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:33.935565 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 03:35:33.935642 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:35:33.935812 14161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.930961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:41963 (local address 127.1.224.66:57165)
0809 03:35:33.931068 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.931083 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:33.931169 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:33.931478 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:33.931481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:33.931499 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:33.931634 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.931642 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.932720 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.932723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.934263 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.934272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.935197 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.935201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.935408 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.935448 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 03:35:33.935530 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 03:35:33.935649 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":151}
I20260809 03:35:33.935812 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.930572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33961 (local address 127.1.224.66:34577)
0809 03:35:33.931001 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.931019 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:33.931585 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:33.931888 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:33.931890 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:33.931907 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:33.931991 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.931996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.932905 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.932908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.934260 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:33.934272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:33.935197 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:33.935201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:33.935412 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:33.935453 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 03:35:33.935531 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 03:35:33.935649 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":176}
I20260809 03:35:33.936630 13961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "329e55b0de274cd6abebbe76dd17f835" candidate_uuid: "eab5b9e49845493fb3bab869bc385008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4f7a00046b74bffb36c8796ffa67be1" is_pre_election: true
I20260809 03:35:33.936721 13537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "329e55b0de274cd6abebbe76dd17f835" candidate_uuid: "eab5b9e49845493fb3bab869bc385008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca54a7b094841a5883552c27c0e94c0" is_pre_election: true
I20260809 03:35:33.936861 13537 raft_consensus.cc:2454] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eab5b9e49845493fb3bab869bc385008 in term 0.
I20260809 03:35:33.936908 13961 raft_consensus.cc:2454] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eab5b9e49845493fb3bab869bc385008 in term 0.
I20260809 03:35:33.937310 13650 leader_election.cc:302] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4f7a00046b74bffb36c8796ffa67be1, eab5b9e49845493fb3bab869bc385008; no voters:
I20260809 03:35:33.937582 14156 raft_consensus.cc:2781] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:33.937647 14156 raft_consensus.cc:482] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:33.937686 14156 raft_consensus.cc:3037] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:33.938791 14156 raft_consensus.cc:504] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } } peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } }
I20260809 03:35:33.939158 14156 leader_election.cc:288] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [CANDIDATE]: Term 1 election: Requested vote from peers bca54a7b094841a5883552c27c0e94c0 (127.1.224.65:33961), a4f7a00046b74bffb36c8796ffa67be1 (127.1.224.67:41963)
I20260809 03:35:33.939550 13537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "329e55b0de274cd6abebbe76dd17f835" candidate_uuid: "eab5b9e49845493fb3bab869bc385008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca54a7b094841a5883552c27c0e94c0"
I20260809 03:35:33.939579 13961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "329e55b0de274cd6abebbe76dd17f835" candidate_uuid: "eab5b9e49845493fb3bab869bc385008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4f7a00046b74bffb36c8796ffa67be1"
I20260809 03:35:33.939685 13961 raft_consensus.cc:3037] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:33.939685 13537 raft_consensus.cc:3037] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:33.940861 13961 raft_consensus.cc:2454] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eab5b9e49845493fb3bab869bc385008 in term 1.
I20260809 03:35:33.940861 13537 raft_consensus.cc:2454] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eab5b9e49845493fb3bab869bc385008 in term 1.
I20260809 03:35:33.941203 13650 leader_election.cc:302] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4f7a00046b74bffb36c8796ffa67be1, eab5b9e49845493fb3bab869bc385008; no voters:
I20260809 03:35:33.941380 14156 raft_consensus.cc:2781] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:33.941605 14156 raft_consensus.cc:686] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 1 LEADER]: Becoming Leader. State: Replica: eab5b9e49845493fb3bab869bc385008, State: Running, Role: LEADER
I20260809 03:35:33.942328 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.942095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.66:37245)
0809 03:35:33.942201 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.942258 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 03:35:33.942293 14156 consensus_queue.cc:237] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } } peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } }
I20260809 03:35:33.945084 13024 catalog_manager.cc:5495] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 reported cstate change: term changed from 0 to 1, leader changed from <none> to eab5b9e49845493fb3bab869bc385008 (127.1.224.66). New cstate: current_term: 1 leader_uuid: "eab5b9e49845493fb3bab869bc385008" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eab5b9e49845493fb3bab869bc385008" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46675 } health_report { overall_health: HEALTHY } } }
I20260809 03:35:33.964715 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.964266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55696)
0809 03:35:33.964557 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.964619 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":88,"threads_started":1}
I20260809 03:35:33.965525 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:33.965289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.0.0.1:55708)
0809 03:35:33.965412 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:33.965465 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 03:35:34.030951 14171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:34.026721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43414 (local address 127.1.224.66:46675)
0809 03:35:34.026994 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:34.027015 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:34.027029 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:35:34.027126 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:34.027128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:34.027188 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:34.027297 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:34.027571 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:34.028108 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:34.029170 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:34.029427 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:34.029522 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:34.029816 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:34.030350 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:34.030450 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:34.030594 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 03:35:34.030654 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":100,"threads_started":1}
I20260809 03:35:34.034123 13537 raft_consensus.cc:1261] T 329e55b0de274cd6abebbe76dd17f835 P bca54a7b094841a5883552c27c0e94c0 [term 1 FOLLOWER]: Refusing update from remote peer eab5b9e49845493fb3bab869bc385008: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:34.034480 13961 raft_consensus.cc:1261] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1 [term 1 FOLLOWER]: Refusing update from remote peer eab5b9e49845493fb3bab869bc385008: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:34.034775 14156 consensus_queue.cc:1035] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca54a7b094841a5883552c27c0e94c0" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:34.035180 14164 consensus_queue.cc:1035] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4f7a00046b74bffb36c8796ffa67be1" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:34.037631 14180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:34.036875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.67:41521)
0809 03:35:34.037452 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:34.037525 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":61,"threads_started":1}
I20260809 03:35:34.039827 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:34.039525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.66:36955)
0809 03:35:34.039682 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:34.039748 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 03:35:34.043208 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:34.042941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.66:46191)
0809 03:35:34.043085 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:34.043136 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 03:35:34.044343 14180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:34.043999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.67:54295)
0809 03:35:34.044191 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:34.044254 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 03:35:34.044973 14177 mvcc.cc:205] Tried to move back new op lower bound from 7316465803398086656 to 7316465803031072768. Current Snapshot: MvccSnapshot[applied={T|T < 7316465803398086656 or (T in {7316465803398086656})}]
I20260809 03:35:34.045320 14179 mvcc.cc:205] Tried to move back new op lower bound from 7316465803398086656 to 7316465803031072768. Current Snapshot: MvccSnapshot[applied={T|T < 7316465803398086656}]
I20260809 03:35:34.045763 14182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:34.037439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.65:41053)
0809 03:35:34.045604 (+ 8165us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:34.045665 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8090,"thread_start_us":62,"threads_started":1}
I20260809 03:35:34.046733 14191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:34.046296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39163 (local address 127.1.224.65:38085)
0809 03:35:34.046580 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:34.046647 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:39163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":121,"threads_started":1}
I20260809 03:35:34.049397 13033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:34.049578 13033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:34.053653 13033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9abcb73ba3c448b79a20e82f3bf2b131: Sending DeleteTablet for 3 replicas of tablet 329e55b0de274cd6abebbe76dd17f835
I20260809 03:35:34.054482 13931 tablet_service.cc:1514] Processing DeleteTablet for tablet 329e55b0de274cd6abebbe76dd17f835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:34 UTC) from {username='oryx', principal='oryx/127.1.224.125@KRBTEST.COM'} at 127.0.0.1:59242
I20260809 03:35:34.054482 13505 tablet_service.cc:1514] Processing DeleteTablet for tablet 329e55b0de274cd6abebbe76dd17f835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:34 UTC) from {username='oryx', principal='oryx/127.1.224.125@KRBTEST.COM'} at 127.0.0.1:40258
I20260809 03:35:34.054553 13720 tablet_service.cc:1514] Processing DeleteTablet for tablet 329e55b0de274cd6abebbe76dd17f835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:34 UTC) from {username='oryx', principal='oryx/127.1.224.125@KRBTEST.COM'} at 127.0.0.1:43406
I20260809 03:35:34.054944 14195 tablet_replica.cc:330] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008: stopping tablet replica
I20260809 03:35:34.055102 14195 raft_consensus.cc:2229] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:34.055444 14195 raft_consensus.cc:2258] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:34.056034 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 13256
I20260809 03:35:34.057001 14195 ts_tablet_manager.cc:1905] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:34.060179 14193 tablet_replica.cc:330] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1: stopping tablet replica
I20260809 03:35:34.060204 14195 ts_tablet_manager.cc:1918] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:34.060269 14195 log.cc:1196] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/329e55b0de274cd6abebbe76dd17f835
I20260809 03:35:34.060333 14193 raft_consensus.cc:2229] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:34.060523 14193 raft_consensus.cc:2258] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:34.060616 14195 ts_tablet_manager.cc:1939] T 329e55b0de274cd6abebbe76dd17f835 P eab5b9e49845493fb3bab869bc385008: Deleting consensus metadata
I20260809 03:35:34.061237 14193 ts_tablet_manager.cc:1905] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:34.063845 14193 ts_tablet_manager.cc:1918] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:34.063905 14193 log.cc:1196] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/329e55b0de274cd6abebbe76dd17f835
I20260809 03:35:34.064193 14193 ts_tablet_manager.cc:1939] T 329e55b0de274cd6abebbe76dd17f835 P a4f7a00046b74bffb36c8796ffa67be1: Deleting consensus metadata
I20260809 03:35:34.065006 13018 catalog_manager.cc:4841] TS a4f7a00046b74bffb36c8796ffa67be1 (127.1.224.67:41963): tablet 329e55b0de274cd6abebbe76dd17f835 (table test-table [id=f7e2c079841f4123b8bc637103f3abd3]) successfully deleted
I20260809 03:35:34.065805 13014 catalog_manager.cc:4841] TS eab5b9e49845493fb3bab869bc385008 (127.1.224.66:46675): tablet 329e55b0de274cd6abebbe76dd17f835 (table test-table [id=f7e2c079841f4123b8bc637103f3abd3]) successfully deleted
I20260809 03:35:34.072563 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 13619
I20260809 03:35:34.072595 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:34.072253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33961 (local address 127.0.0.1:40270)
0809 03:35:34.072407 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:34.072496 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:33961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 03:35:34.081894 13018 catalog_manager.cc:4584] TS bca54a7b094841a5883552c27c0e94c0 (127.1.224.65:33961): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 329e55b0de274cd6abebbe76dd17f835: Network error: Client connection negotiation failed: client connection to 127.1.224.65:33961: connect: Connection refused (error 111)
I20260809 03:35:34.090390 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 13832
I20260809 03:35:34.107493 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 12830
I20260809 03:35:34.124541 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 12982
2026-08-09T03:35:34Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6654 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:34 dist-test-slave-jq4t krb5kdc[14201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:34 dist-test-slave-jq4t krb5kdc[14201](info): set up 2 sockets
Aug 09 03:35:34 dist-test-slave-jq4t krb5kdc[14201](info): commencing operation
krb5kdc: starting...
W20260809 03:35:35.387742 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:35:35 dist-test-slave-jq4t krb5kdc[14201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.126.keytab.
Entry for principal oryx/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.126.keytab.
I20260809 03:35:35.532830 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:41345
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.126.keytab
--principal=oryx/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:40099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:41345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:35.652211 14219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:35.652529 14219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:35.652573 14219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:35.656734 14219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:35.656800 14219 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 03:35:35.656818 14219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:35.656843 14219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:35.656860 14219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:35.656926 14219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:35.662508 14219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:40099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:41345
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.126.keytab
--principal=oryx/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:41345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:35.663813 14219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:35.664953 14219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:35.674023 14229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:35.674029 14232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:35.674198 14219 server_base.cc:1034] running on GCE node
W20260809 03:35:35.674077 14228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:35.674952 14219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:35.676128 14219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:35.677325 14219 hybrid_clock.cc:648] HybridClock initialized: now 1786246535677305 us; error 34 us; skew 500 ppm
Aug 09 03:35:35 dist-test-slave-jq4t krb5kdc[14201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246535, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:35.683904 14219 init.cc:377] Logged in from keytab as oryx/127.1.224.126@KRBTEST.COM (short username oryx)
I20260809 03:35:35.685048 14219 webserver.cc:460] Webserver started at http://127.1.224.126:46383/ using document root <none> and password file <none>
I20260809 03:35:35.685704 14219 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:35.685807 14219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:35.686076 14219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:35.688428 14219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "4c0475a604794988a8e8514cdae680f1"
format_stamp: "Formatted at 2026-08-09 03:35:35 on dist-test-slave-jq4t"
server_key: "7eb7eefc53bac67d179f1ccad854dda8"
server_key_iv: "19ce76554b7cf99c050696fc97ca7a52"
server_key_version: "encryptionkey@0"
I20260809 03:35:35.689114 14219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "4c0475a604794988a8e8514cdae680f1"
format_stamp: "Formatted at 2026-08-09 03:35:35 on dist-test-slave-jq4t"
server_key: "7eb7eefc53bac67d179f1ccad854dda8"
server_key_iv: "19ce76554b7cf99c050696fc97ca7a52"
server_key_version: "encryptionkey@0"
I20260809 03:35:35.693392 14219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:35:35.695959 14243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:35.697319 14219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:35:35.697491 14219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "4c0475a604794988a8e8514cdae680f1"
format_stamp: "Formatted at 2026-08-09 03:35:35 on dist-test-slave-jq4t"
server_key: "7eb7eefc53bac67d179f1ccad854dda8"
server_key_iv: "19ce76554b7cf99c050696fc97ca7a52"
server_key_version: "encryptionkey@0"
I20260809 03:35:35.697616 14219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:35.727726 14219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:35.731756 14219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:35.732017 14219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:35.741499 14219 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:41345
I20260809 03:35:35.741524 14323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:41345 every 8 connection(s)
I20260809 03:35:35.742641 14219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:35.745743 14324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:35.751398 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 14219
I20260809 03:35:35.751576 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:35.751782 14324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1: Bootstrap starting.
I20260809 03:35:35.751892 1921 external_mini_cluster.cc:1428] Setting key 549dc4d67990ec573db536e0f27ef782
I20260809 03:35:35.754503 14324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:35.755379 14324 log.cc:824] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:35.757499 14324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1: No bootstrap required, opened a new log
Aug 09 03:35:35 dist-test-slave-jq4t krb5kdc[14201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:35.761824 14324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c0475a604794988a8e8514cdae680f1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41345 } }
I20260809 03:35:35.762126 14324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:35.762202 14324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c0475a604794988a8e8514cdae680f1, State: Initialized, Role: FOLLOWER
I20260809 03:35:35.762706 14324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c0475a604794988a8e8514cdae680f1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41345 } }
I20260809 03:35:35.762892 14324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:35.763007 14324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:35.763157 14324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:35.764227 14324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c0475a604794988a8e8514cdae680f1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41345 } }
I20260809 03:35:35.764649 14324 leader_election.cc:302] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c0475a604794988a8e8514cdae680f1; no voters:
I20260809 03:35:35.765056 14324 leader_election.cc:288] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:35.765147 14333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:35.765357 14333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [term 1 LEADER]: Becoming Leader. State: Replica: 4c0475a604794988a8e8514cdae680f1, State: Running, Role: LEADER
I20260809 03:35:35.765683 14333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c0475a604794988a8e8514cdae680f1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41345 } }
I20260809 03:35:35.766502 14324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:35.767381 14336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c0475a604794988a8e8514cdae680f1. Latest consensus state: current_term: 1 leader_uuid: "4c0475a604794988a8e8514cdae680f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c0475a604794988a8e8514cdae680f1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41345 } } }
I20260809 03:35:35.767521 14336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:35.767760 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:35.753350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36640 (local address 127.1.224.126:41345)
0809 03:35:35.753807 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:35.753832 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:35.753870 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:35:35.754534 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:35.754584 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:35.754951 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:35.755236 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:35.756200 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:35.757281 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:35.758213 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:35.758694 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:35.758870 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:35.761769 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:35.761774 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:35.761788 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:35.761816 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:35.763763 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:35.764293 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:35.764299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:35.764306 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:35.764457 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:35.764797 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:35.764800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:35.764801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:35.765373 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:35.766202 (+ 829us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:35.766357 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 03:35:35.766695 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":154,"threads_started":1}
I20260809 03:35:35.767841 14334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c0475a604794988a8e8514cdae680f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c0475a604794988a8e8514cdae680f1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41345 } } }
I20260809 03:35:35.767946 14334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1 [sys.catalog]: This master's current role is: LEADER
W20260809 03:35:35.771701 14353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:35:35.771790 14353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:35:35.771869 14343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:35.772790 14343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:35.778631 14343 catalog_manager.cc:1347] Generated new cluster ID: 2d8961e1deb4457590ccbf8e20a4ab70
I20260809 03:35:35.778716 14343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:35.789105 14343 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:35.790084 14343 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:35.809993 14343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1: Generated new TSK 0
I20260809 03:35:35.810855 14343 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab.
Entry for principal oryx/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab.
I20260809 03:35:35.848098 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:41345
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab
--principal=oryx/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:40099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:35.970486 14366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:35.970789 14366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:35.970889 14366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:35.975059 14366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:35.975163 14366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:35.975237 14366 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:35:35.975306 14366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:35.980895 14366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:40099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab
--principal=oryx/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.224.126:41345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:35.982167 14366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:35.983312 14366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:35.993597 14372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:35.993604 14371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:35.993904 14366 server_base.cc:1034] running on GCE node
W20260809 03:35:35.993604 14374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:35.994539 14366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:35.995167 14366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:35.996344 14366 hybrid_clock.cc:648] HybridClock initialized: now 1786246535996317 us; error 48 us; skew 500 ppm
Aug 09 03:35:35 dist-test-slave-jq4t krb5kdc[14201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246535, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:35.999295 14366 init.cc:377] Logged in from keytab as oryx/127.1.224.65@KRBTEST.COM (short username oryx)
I20260809 03:35:36.000519 14366 webserver.cc:460] Webserver started at http://127.1.224.65:46601/ using document root <none> and password file <none>
I20260809 03:35:36.001184 14366 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:36.001283 14366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:36.001544 14366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:36.003772 14366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "fcacbc2374f245f5b446b989a7c0b1f7"
format_stamp: "Formatted at 2026-08-09 03:35:35 on dist-test-slave-jq4t"
server_key: "d1c1f55c464b91293e14e8b7416ecec1"
server_key_iv: "6d3d0203aa4a335f4c597ae2bfec30b7"
server_key_version: "encryptionkey@0"
I20260809 03:35:36.004392 14366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "fcacbc2374f245f5b446b989a7c0b1f7"
format_stamp: "Formatted at 2026-08-09 03:35:35 on dist-test-slave-jq4t"
server_key: "d1c1f55c464b91293e14e8b7416ecec1"
server_key_iv: "6d3d0203aa4a335f4c597ae2bfec30b7"
server_key_version: "encryptionkey@0"
I20260809 03:35:36.008605 14366 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:35:36.011363 14384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:36.012691 14366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:36.012832 14366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "fcacbc2374f245f5b446b989a7c0b1f7"
format_stamp: "Formatted at 2026-08-09 03:35:35 on dist-test-slave-jq4t"
server_key: "d1c1f55c464b91293e14e8b7416ecec1"
server_key_iv: "6d3d0203aa4a335f4c597ae2bfec30b7"
server_key_version: "encryptionkey@0"
I20260809 03:35:36.012964 14366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:36.022994 14366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:36.026512 14366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:36.026739 14366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:36.028716 14366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:36.028800 14366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:36.028875 14366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:36.028932 14366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:35:36 dist-test-slave-jq4t krb5kdc[14201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246535, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:36.040520 14366 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:39365
I20260809 03:35:36.040621 14547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:39365 every 8 connection(s)
I20260809 03:35:36.041823 14366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:36.044238 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.029462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:42057 (local address 127.1.224.126:41345)
0809 03:35:36.029629 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.029634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.030616 (+ 982us) server_negotiation.cc:407] Connection header received
0809 03:35:36.031628 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.031631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.031692 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.031783 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:36.033751 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.034464 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.036462 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.036762 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.036893 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.039980 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:36.039986 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:36.039991 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:36.040035 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:36.041953 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.042660 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.042664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.042668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.042774 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.043170 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.043173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.043174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.043388 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:36.043473 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.043972 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.044107 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:35:36.045094 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.029927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:41345 (local address 127.1.224.65:42057)
0809 03:35:36.030411 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.030466 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.031399 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.032080 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.032092 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.032697 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:36.033528 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.033554 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.034610 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.034617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.036292 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.036309 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.036947 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.036951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.037161 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.037870 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:36.037887 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:36.039755 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:36.042086 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.042092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.042105 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.042482 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.042894 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.042898 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.042900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.043066 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.043491 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:36.043496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:36.043753 (+ 257us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.043923 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.044191 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":127,"threads_started":1}
I20260809 03:35:36.046444 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 14366
I20260809 03:35:36.046494 14548 heartbeater.cc:347] Connected to a master server at 127.1.224.126:41345
I20260809 03:35:36.046590 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:36.046744 14548 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:36.046954 1921 external_mini_cluster.cc:1428] Setting key fbebdf766c61bb03143ec29d6b44e4eb
I20260809 03:35:36.047366 14548 heartbeater.cc:511] Master 127.1.224.126:41345 requested a full tablet report, sending...
I20260809 03:35:36.049204 14265 ts_manager.cc:194] Registered new tserver with Master: fcacbc2374f245f5b446b989a7c0b1f7 (127.1.224.65:39365)
I20260809 03:35:36.050400 14265 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:42057
WARNING: no policy specified for oryx/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:35:36.059470 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.051010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:54519 (local address 127.1.224.126:41345)
0809 03:35:36.051163 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.051169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.051183 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:35:36.051224 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.051229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.051281 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.051359 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:36.052110 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.052911 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.054266 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.054506 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.054599 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.055871 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:36.055875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:36.055878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:36.055912 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:36.057537 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.058019 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.058022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.058024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.058113 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.058466 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.058469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.058471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.058738 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:36.058853 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.059151 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.059281 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:35:36.059518 14561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.049180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:41345 (local address 127.1.224.65:54519)
0809 03:35:36.050749 (+ 1569us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.050769 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.050889 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.051500 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.051504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.051777 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:36.051947 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.051959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.053054 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.053060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.054124 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.054141 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.054640 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.054644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.054735 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.055357 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:36.055370 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:36.055718 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:36.057666 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.057670 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.057673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.057898 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.058211 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.058215 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.058217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.058332 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.058844 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:36.058847 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:36.058970 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.059093 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.059277 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1449,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab.
Entry for principal oryx/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab.
I20260809 03:35:36.080251 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:41345
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab
--principal=oryx/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:40099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:36.198815 14564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:36.199172 14564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:36.199257 14564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:36.203608 14564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:36.203725 14564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:36.203804 14564 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:35:36.203889 14564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:36.209790 14564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:40099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab
--principal=oryx/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.224.126:41345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:36.211105 14564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:36.212248 14564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:36.222729 14574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:36.222885 14564 server_base.cc:1034] running on GCE node
W20260809 03:35:36.222750 14569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:36.222729 14570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:36.223423 14564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:36.224167 14564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:36.225327 14564 hybrid_clock.cc:648] HybridClock initialized: now 1786246536225293 us; error 48 us; skew 500 ppm
Aug 09 03:35:36 dist-test-slave-jq4t krb5kdc[14201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246536, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:36.228334 14564 init.cc:377] Logged in from keytab as oryx/127.1.224.66@KRBTEST.COM (short username oryx)
I20260809 03:35:36.229306 14564 webserver.cc:460] Webserver started at http://127.1.224.66:35311/ using document root <none> and password file <none>
I20260809 03:35:36.229902 14564 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:36.229966 14564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:36.230145 14564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:36.232326 14564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "21035c20fcd643f8ab869e65744768c9"
format_stamp: "Formatted at 2026-08-09 03:35:36 on dist-test-slave-jq4t"
server_key: "3861a9f4212954f63d1150a13bab0326"
server_key_iv: "d799514d2e146dc4a0d82c94287b7ad5"
server_key_version: "encryptionkey@0"
I20260809 03:35:36.233039 14564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "21035c20fcd643f8ab869e65744768c9"
format_stamp: "Formatted at 2026-08-09 03:35:36 on dist-test-slave-jq4t"
server_key: "3861a9f4212954f63d1150a13bab0326"
server_key_iv: "d799514d2e146dc4a0d82c94287b7ad5"
server_key_version: "encryptionkey@0"
I20260809 03:35:36.237087 14564 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:35:36.239745 14591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:36.241199 14564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:36.241358 14564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "21035c20fcd643f8ab869e65744768c9"
format_stamp: "Formatted at 2026-08-09 03:35:36 on dist-test-slave-jq4t"
server_key: "3861a9f4212954f63d1150a13bab0326"
server_key_iv: "d799514d2e146dc4a0d82c94287b7ad5"
server_key_version: "encryptionkey@0"
I20260809 03:35:36.241493 14564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:36.268698 14564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:36.272059 14564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:36.272279 14564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:36.273938 14564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:36.274022 14564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:36.274103 14564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:36.274147 14564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:35:36 dist-test-slave-jq4t krb5kdc[14201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246536, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:36.285771 14564 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:46611
I20260809 03:35:36.286167 14777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:46611 every 8 connection(s)
I20260809 03:35:36.286939 14564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:36.288780 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 14564
I20260809 03:35:36.288925 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:36.289283 1921 external_mini_cluster.cc:1428] Setting key 124b83de0b037edc173b7a8b1181290c
I20260809 03:35:36.289729 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.275057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:47747 (local address 127.1.224.126:41345)
0809 03:35:36.275201 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.275205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.276414 (+ 1209us) server_negotiation.cc:407] Connection header received
0809 03:35:36.277676 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.277680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.277745 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.277838 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:36.279743 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.280689 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.282119 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.282368 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.282493 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.285409 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:36.285413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:36.285416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:36.285464 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:36.287212 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.287853 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.287856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.287857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.287985 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.288342 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.288345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.288346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.288644 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:36.288735 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.289487 (+ 752us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.289604 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:35:36.290817 14610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.275641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:41345 (local address 127.1.224.66:47747)
0809 03:35:36.276232 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.276282 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.277425 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.278108 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.278120 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.278692 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:36.279531 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.279559 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.280831 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.280835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.281987 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.281998 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.282545 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.282549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.282729 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.283414 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:36.283433 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:36.285227 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:36.287333 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.287339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.287355 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.287721 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.288089 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.288092 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.288093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.288237 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.288767 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:36.288776 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:36.289157 (+ 381us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.289385 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.289761 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":158,"threads_started":1}
I20260809 03:35:36.292251 14779 heartbeater.cc:347] Connected to a master server at 127.1.224.126:41345
I20260809 03:35:36.292407 14779 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:36.293071 14779 heartbeater.cc:511] Master 127.1.224.126:41345 requested a full tablet report, sending...
I20260809 03:35:36.294301 14265 ts_manager.cc:194] Registered new tserver with Master: 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611)
I20260809 03:35:36.295090 14265 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:47747
WARNING: no policy specified for oryx/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:35:36.302915 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.294753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:52095 (local address 127.1.224.126:41345)
0809 03:35:36.294873 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.294877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.295226 (+ 349us) server_negotiation.cc:407] Connection header received
0809 03:35:36.295415 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.295418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.295464 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.295566 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:36.296178 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.296953 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.297869 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.298111 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.298227 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.299434 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:36.299439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:36.299441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:36.299474 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:36.301056 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.301557 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.301561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.301564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.301647 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.301962 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.301966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.301968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.302215 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:36.302340 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.302632 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.302751 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 03:35:36.302927 14789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.294701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:41345 (local address 127.1.224.66:52095)
0809 03:35:36.295139 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.295156 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.295302 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.295612 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.295617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.295862 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:36.296038 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.296048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.297112 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.297116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.297736 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.297745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.298233 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.298236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.298337 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.298951 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:36.298968 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:36.299292 (+ 324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:36.301181 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.301184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.301187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.301436 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.301745 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.301748 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.301750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.301843 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.302346 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:36.302352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:36.302455 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.302573 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.302739 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab.
Entry for principal oryx/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab.
I20260809 03:35:36.323957 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:41345
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab
--principal=oryx/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:40099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:36.443831 14791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:36.444136 14791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:36.444185 14791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:36.448768 14791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:36.448840 14791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:36.448881 14791 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:35:36.448942 14791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:36.454496 14791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:40099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab
--principal=oryx/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.224.126:41345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:36.455673 14791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:36.456794 14791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:36.467060 14800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:36.467065 14799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:36.467065 14804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:36.467406 14791 server_base.cc:1034] running on GCE node
I20260809 03:35:36.467970 14791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:36.468760 14791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:36.469962 14791 hybrid_clock.cc:648] HybridClock initialized: now 1786246536469937 us; error 57 us; skew 500 ppm
Aug 09 03:35:36 dist-test-slave-jq4t krb5kdc[14201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246536, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:36.472903 14791 init.cc:377] Logged in from keytab as oryx/127.1.224.67@KRBTEST.COM (short username oryx)
I20260809 03:35:36.473776 14791 webserver.cc:460] Webserver started at http://127.1.224.67:42477/ using document root <none> and password file <none>
I20260809 03:35:36.474321 14791 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:36.474380 14791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:36.474560 14791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:36.476698 14791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "2e863fae3a4b4eff91ec29946bd1154d"
format_stamp: "Formatted at 2026-08-09 03:35:36 on dist-test-slave-jq4t"
server_key: "621612e79e4e58a7148ef951fa68aacd"
server_key_iv: "654e7cb32eca5da065c038a8de3ba59d"
server_key_version: "encryptionkey@0"
I20260809 03:35:36.477231 14791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "2e863fae3a4b4eff91ec29946bd1154d"
format_stamp: "Formatted at 2026-08-09 03:35:36 on dist-test-slave-jq4t"
server_key: "621612e79e4e58a7148ef951fa68aacd"
server_key_iv: "654e7cb32eca5da065c038a8de3ba59d"
server_key_version: "encryptionkey@0"
I20260809 03:35:36.481024 14791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:35:36.483490 14818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:36.484838 14791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:36.484972 14791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "2e863fae3a4b4eff91ec29946bd1154d"
format_stamp: "Formatted at 2026-08-09 03:35:36 on dist-test-slave-jq4t"
server_key: "621612e79e4e58a7148ef951fa68aacd"
server_key_iv: "654e7cb32eca5da065c038a8de3ba59d"
server_key_version: "encryptionkey@0"
I20260809 03:35:36.485086 14791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:36.496282 14791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:36.499800 14791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:36.499970 14791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:36.501667 14791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:36.501726 14791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:36.501804 14791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:36.501844 14791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:35:36 dist-test-slave-jq4t krb5kdc[14201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246536, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:36.513475 14791 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:41243
I20260809 03:35:36.513504 14994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:41243 every 8 connection(s)
I20260809 03:35:36.514520 14791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:36.517999 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.502601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:47515 (local address 127.1.224.126:41345)
0809 03:35:36.502767 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.502773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.504656 (+ 1883us) server_negotiation.cc:407] Connection header received
0809 03:35:36.505708 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.505711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.505775 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.505896 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:36.507642 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.508593 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.510465 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.510720 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.510859 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.514071 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:36.514075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:36.514077 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:36.514105 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:36.515812 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.516400 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.516403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.516404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.516548 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.516879 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.516881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.516883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.517085 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:36.517167 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.517749 (+ 582us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.517868 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 03:35:36.518854 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.503077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:41345 (local address 127.1.224.67:47515)
0809 03:35:36.504427 (+ 1350us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.504517 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.505476 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.506124 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.506133 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.506654 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:36.507453 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.507474 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.508747 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.508750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.510320 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.510332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.510907 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.510912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.511116 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.511770 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:36.511787 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:36.513854 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:36.515923 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.515928 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.515946 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.516301 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.516646 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.516649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.516650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.516793 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.517181 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:36.517187 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:36.517477 (+ 290us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.517702 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.517989 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1099,"thread_start_us":133,"threads_started":1}
I20260809 03:35:36.520198 14995 heartbeater.cc:347] Connected to a master server at 127.1.224.126:41345
I20260809 03:35:36.520375 14995 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:36.520963 14995 heartbeater.cc:511] Master 127.1.224.126:41345 requested a full tablet report, sending...
I20260809 03:35:36.522136 14265 ts_manager.cc:194] Registered new tserver with Master: 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67:41243)
I20260809 03:35:36.522356 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 14791
I20260809 03:35:36.522490 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:36.522840 1921 external_mini_cluster.cc:1428] Setting key 483c38cdb464728d3ea4d37bd04280e7
I20260809 03:35:36.522980 14265 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:47515
I20260809 03:35:36.525192 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:35:36.531167 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.522740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:38399 (local address 127.1.224.126:41345)
0809 03:35:36.522873 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.522880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.523116 (+ 236us) server_negotiation.cc:407] Connection header received
0809 03:35:36.523343 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.523347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.523398 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.523494 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:36.524228 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.524998 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.525973 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.526355 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.526491 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.527677 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:36.527680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:36.527683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:36.527718 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:36.529321 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.529816 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.529820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.529822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.529917 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.530221 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.530225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.530227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.530444 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:36.530560 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.530849 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.530997 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:35:36.531179 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.522617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:41345 (local address 127.1.224.67:38399)
0809 03:35:36.523009 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.523028 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.523194 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.523560 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.523562 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.523932 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:36.524100 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.524109 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.525146 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.525150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.525836 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.525847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.526484 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.526487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.526558 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.527165 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:36.527178 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:36.527522 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:36.529464 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.529467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.529478 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.529695 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.530033 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.530036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.530038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.530125 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.530548 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:36.530550 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:36.530658 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.530770 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.530968 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":129,"threads_started":1}
Aug 09 03:35:36 dist-test-slave-jq4t krb5kdc[14201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246536, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:35:36 dist-test-slave-jq4t krb5kdc[14201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:36 dist-test-slave-jq4t krb5kdc[14201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:35:36.553289 14330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:36.546896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36646 (local address 127.1.224.126:41345)
0809 03:35:36.547063 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.547067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.547081 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:35:36.547125 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.547127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.547170 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.547273 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:36.548033 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.548641 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.549497 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.549718 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.549798 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.553119 (+ 3321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36646: BlockingRecv error: recv got EOF from 127.0.0.1:36646 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 03:35:36 dist-test-slave-jq4t krb5kdc[14201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246536, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:36.564123 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.555268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36658 (local address 127.1.224.126:41345)
0809 03:35:36.555432 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.555439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.555512 (+ 73us) server_negotiation.cc:407] Connection header received
0809 03:35:36.555689 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.555693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.555747 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.555847 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:36.556525 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.557055 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.557867 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.558101 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.558213 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.560905 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:36.560908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:36.560910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:36.560937 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:36.562501 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.562957 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.562960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.562961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.563056 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.563358 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.563360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.563361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.563554 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:36.563658 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.563885 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.563994 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 03:35:36.567763 14265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:36.571363 14265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:36.586699 15029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.580640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53986 (local address 127.1.224.67:41243)
0809 03:35:36.581101 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.581325 (+ 224us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.581685 (+ 360us) server_negotiation.cc:407] Connection header received
0809 03:35:36.581773 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.581779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.582017 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.582177 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:36.582494 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.583381 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.585075 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.585930 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.586048 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.586230 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.586411 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.586530 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":109,"threads_started":1}
I20260809 03:35:36.586701 15027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.580370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:41243 (local address 127.0.0.1:53986)
0809 03:35:36.581457 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.581468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.581583 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.582193 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.582199 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.582222 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:36.582380 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.582390 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.583479 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.583482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.584925 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.584937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.586074 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.586077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.586192 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.586237 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.586363 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.586519 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"mutex_wait_us":89,"spinlock_wait_cycles":11136,"thread_start_us":142,"threads_started":1}
I20260809 03:35:36.586954 15026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.580370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:46611 (local address 127.0.0.1:51952)
0809 03:35:36.580804 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.580859 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.581007 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.583216 (+ 2209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.583219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.583239 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:36.583325 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.583330 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.584451 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.584455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.585560 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.585570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.586577 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.586581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.586669 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.586692 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.586744 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.586821 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":29,"thread_start_us":118,"threads_started":1}
I20260809 03:35:36.586706 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.581182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39365 (local address 127.0.0.1:36256)
0809 03:35:36.581765 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.581780 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.581885 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.582365 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.582368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.582387 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:36.582498 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.582504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.583893 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.583896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.585056 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.585064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.586285 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.586295 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.586409 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.586436 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.586510 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.586577 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":132,"threads_started":1}
I20260809 03:35:36.587039 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.581395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36256 (local address 127.1.224.65:39365)
0809 03:35:36.581702 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.581710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.581873 (+ 163us) server_negotiation.cc:407] Connection header received
0809 03:35:36.581996 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.582001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.582174 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.582297 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:36.582620 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.583761 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.585186 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.585936 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.586048 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.586212 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.586783 (+ 571us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.586917 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":87,"threads_started":1}
I20260809 03:35:36.587710 15032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.581825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51952 (local address 127.1.224.66:46611)
0809 03:35:36.582777 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.582789 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.582811 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:35:36.582883 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.582888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.583061 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.583209 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:36.583448 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.584324 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.585691 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.586427 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.587271 (+ 844us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.587432 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.587493 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.587579 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":844,"thread_start_us":121,"threads_started":1}
I20260809 03:35:36.590247 14450 tablet_service.cc:1467] Processing CreateTablet for tablet 8f080f9ffaf94aaf8d356cd7abd7b7de (DEFAULT_TABLE table=test-table [id=c593eccda8e34bb59ae9d021757fe77a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:36.590375 14680 tablet_service.cc:1467] Processing CreateTablet for tablet 8f080f9ffaf94aaf8d356cd7abd7b7de (DEFAULT_TABLE table=test-table [id=c593eccda8e34bb59ae9d021757fe77a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:36.591346 14450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f080f9ffaf94aaf8d356cd7abd7b7de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:36.591357 14680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f080f9ffaf94aaf8d356cd7abd7b7de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:36.597828 15033 tablet_bootstrap.cc:492] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7: Bootstrap starting.
I20260809 03:35:36.598843 15034 tablet_bootstrap.cc:492] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9: Bootstrap starting.
I20260809 03:35:36.599082 14896 tablet_service.cc:1467] Processing CreateTablet for tablet 8f080f9ffaf94aaf8d356cd7abd7b7de (DEFAULT_TABLE table=test-table [id=c593eccda8e34bb59ae9d021757fe77a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:36.600098 15033 tablet_bootstrap.cc:654] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:36.600256 14896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f080f9ffaf94aaf8d356cd7abd7b7de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:36.600968 15034 tablet_bootstrap.cc:654] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:36.600981 15033 log.cc:824] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:36.601853 15034 log.cc:824] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:36.603336 15033 tablet_bootstrap.cc:492] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7: No bootstrap required, opened a new log
I20260809 03:35:36.603581 15033 ts_tablet_manager.cc:1397] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:35:36.604110 15034 tablet_bootstrap.cc:492] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9: No bootstrap required, opened a new log
I20260809 03:35:36.604341 15034 ts_tablet_manager.cc:1397] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:35:36.607152 15033 raft_consensus.cc:348] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } }
I20260809 03:35:36.607404 15033 raft_consensus.cc:374] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:36.607460 15033 raft_consensus.cc:729] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcacbc2374f245f5b446b989a7c0b1f7, State: Initialized, Role: FOLLOWER
I20260809 03:35:36.607928 15033 consensus_queue.cc:260] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } }
I20260809 03:35:36.608634 14548 heartbeater.cc:503] Master 127.1.224.126:41345 was elected leader, sending a full tablet report...
I20260809 03:35:36.608942 15033 ts_tablet_manager.cc:1428] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 03:35:36.609309 15043 tablet_bootstrap.cc:492] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d: Bootstrap starting.
I20260809 03:35:36.608237 15034 raft_consensus.cc:348] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } }
I20260809 03:35:36.609907 15034 raft_consensus.cc:374] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:36.609968 15034 raft_consensus.cc:729] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21035c20fcd643f8ab869e65744768c9, State: Initialized, Role: FOLLOWER
I20260809 03:35:36.610447 15034 consensus_queue.cc:260] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } }
I20260809 03:35:36.611181 14779 heartbeater.cc:503] Master 127.1.224.126:41345 was elected leader, sending a full tablet report...
I20260809 03:35:36.611575 15034 ts_tablet_manager.cc:1428] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 03:35:36.613838 15043 tablet_bootstrap.cc:654] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:36.614672 15043 log.cc:824] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:36.616670 15043 tablet_bootstrap.cc:492] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d: No bootstrap required, opened a new log
I20260809 03:35:36.616887 15043 ts_tablet_manager.cc:1397] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 03:35:36.620164 15043 raft_consensus.cc:348] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } }
I20260809 03:35:36.620429 15043 raft_consensus.cc:374] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:36.620512 15043 raft_consensus.cc:729] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e863fae3a4b4eff91ec29946bd1154d, State: Initialized, Role: FOLLOWER
I20260809 03:35:36.620949 15043 consensus_queue.cc:260] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } }
I20260809 03:35:36.621687 14995 heartbeater.cc:503] Master 127.1.224.126:41345 was elected leader, sending a full tablet report...
I20260809 03:35:36.621946 15043 ts_tablet_manager.cc:1428] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260809 03:35:36.715700 15047 raft_consensus.cc:482] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:36.715974 15047 raft_consensus.cc:504] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } }
I20260809 03:35:36.717523 15047 leader_election.cc:288] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611), fcacbc2374f245f5b446b989a7c0b1f7 (127.1.224.65:39365)
I20260809 03:35:36.721390 15032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.717959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:36991 (local address 127.1.224.66:46611)
0809 03:35:36.718126 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.718131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.718150 (+ 19us) server_negotiation.cc:407] Connection header received
0809 03:35:36.718217 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.718223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.718287 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.718376 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:36.718718 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.719268 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.720356 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.720797 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.720948 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.721009 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.721183 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.721258 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 03:35:36.721390 14832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.717815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:46611 (local address 127.1.224.67:36991)
0809 03:35:36.717975 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.717992 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.718121 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.718456 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.718459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.718481 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:36.718592 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.718597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.719416 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.719419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.720218 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.720227 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.720934 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.720937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.721043 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.721073 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.721166 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.721249 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 03:35:36.722074 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f080f9ffaf94aaf8d356cd7abd7b7de" candidate_uuid: "2e863fae3a4b4eff91ec29946bd1154d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21035c20fcd643f8ab869e65744768c9" is_pre_election: true
I20260809 03:35:36.722414 14714 raft_consensus.cc:2454] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e863fae3a4b4eff91ec29946bd1154d in term 0.
I20260809 03:35:36.722575 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.718320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:55329 (local address 127.1.224.65:39365)
0809 03:35:36.718453 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.718457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.718597 (+ 140us) server_negotiation.cc:407] Connection header received
0809 03:35:36.718746 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.718749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.718801 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.718885 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:36.719530 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.720119 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.721415 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.721995 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.722143 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.722196 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.722380 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.722458 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:35:36.722568 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.718123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39365 (local address 127.1.224.67:55329)
0809 03:35:36.718494 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.718507 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.718637 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.718897 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.718900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.718916 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:36.719398 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.719405 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.720232 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.720237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.721283 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.721294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.722143 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.722147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.722254 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.722276 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.722349 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.722422 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":139,"threads_started":1}
I20260809 03:35:36.723022 14823 leader_election.cc:302] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21035c20fcd643f8ab869e65744768c9, 2e863fae3a4b4eff91ec29946bd1154d; no voters:
I20260809 03:35:36.723326 15047 raft_consensus.cc:2781] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:36.723320 14477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f080f9ffaf94aaf8d356cd7abd7b7de" candidate_uuid: "2e863fae3a4b4eff91ec29946bd1154d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" is_pre_election: true
I20260809 03:35:36.723434 15047 raft_consensus.cc:482] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:36.723486 15047 raft_consensus.cc:3037] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:36.723627 14477 raft_consensus.cc:2454] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e863fae3a4b4eff91ec29946bd1154d in term 0.
I20260809 03:35:36.724875 15047 raft_consensus.cc:504] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } }
I20260809 03:35:36.725227 15047 leader_election.cc:288] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [CANDIDATE]: Term 1 election: Requested vote from peers 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611), fcacbc2374f245f5b446b989a7c0b1f7 (127.1.224.65:39365)
I20260809 03:35:36.725684 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f080f9ffaf94aaf8d356cd7abd7b7de" candidate_uuid: "2e863fae3a4b4eff91ec29946bd1154d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21035c20fcd643f8ab869e65744768c9"
I20260809 03:35:36.725709 14477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f080f9ffaf94aaf8d356cd7abd7b7de" candidate_uuid: "2e863fae3a4b4eff91ec29946bd1154d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcacbc2374f245f5b446b989a7c0b1f7"
I20260809 03:35:36.725807 14714 raft_consensus.cc:3037] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:36.725853 14477 raft_consensus.cc:3037] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:36.727023 14477 raft_consensus.cc:2454] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e863fae3a4b4eff91ec29946bd1154d in term 1.
I20260809 03:35:36.727347 14714 raft_consensus.cc:2454] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e863fae3a4b4eff91ec29946bd1154d in term 1.
I20260809 03:35:36.727412 14823 leader_election.cc:302] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e863fae3a4b4eff91ec29946bd1154d, fcacbc2374f245f5b446b989a7c0b1f7; no voters:
I20260809 03:35:36.727605 15047 raft_consensus.cc:2781] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:36.727870 15047 raft_consensus.cc:686] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 1 LEADER]: Becoming Leader. State: Replica: 2e863fae3a4b4eff91ec29946bd1154d, State: Running, Role: LEADER
I20260809 03:35:36.728279 15047 consensus_queue.cc:237] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } }
I20260809 03:35:36.732021 14265 catalog_manager.cc:5495] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67). New cstate: current_term: 1 leader_uuid: "2e863fae3a4b4eff91ec29946bd1154d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } health_report { overall_health: UNKNOWN } } }
W20260809 03:35:36.765981 14997 tablet.cc:2369] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:36.777472 15029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.774061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53990 (local address 127.1.224.67:41243)
0809 03:35:36.774191 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.774195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.774208 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:36.774286 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.774288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.774342 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.774438 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:36.774710 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.775224 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.776074 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.776294 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.776370 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.776704 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:36.777157 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:36.777245 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.777310 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.777360 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:35:36.783432 14714 raft_consensus.cc:1261] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Refusing update from remote peer 2e863fae3a4b4eff91ec29946bd1154d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:36.783429 14477 raft_consensus.cc:1261] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Refusing update from remote peer 2e863fae3a4b4eff91ec29946bd1154d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:36.784174 15047 consensus_queue.cc:1035] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [LEADER]: Connected to new peer: Peer: permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:36.784461 15052 consensus_queue.cc:1035] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:35:36.788198 14781 tablet.cc:2369] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:36.793262 15059 mvcc.cc:205] Tried to move back new op lower bound from 7316465814657454080 to 7316465814442536960. Current Snapshot: MvccSnapshot[applied={T|T < 7316465814657454080}]
I20260809 03:35:36.793506 15058 mvcc.cc:205] Tried to move back new op lower bound from 7316465814657454080 to 7316465814442536960. Current Snapshot: MvccSnapshot[applied={T|T < 7316465814657454080}]
I20260809 03:35:36.803476 14265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:36.803674 14265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:36.806223 14265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1: Sending DeleteTablet for 3 replicas of tablet 8f080f9ffaf94aaf8d356cd7abd7b7de
I20260809 03:35:36.807004 14896 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f080f9ffaf94aaf8d356cd7abd7b7de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:36 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:53986
I20260809 03:35:36.807322 14680 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f080f9ffaf94aaf8d356cd7abd7b7de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:36 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:51952
I20260809 03:35:36.807651 14450 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f080f9ffaf94aaf8d356cd7abd7b7de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:36 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:36256
I20260809 03:35:36.808148 15076 tablet_replica.cc:330] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9: stopping tablet replica
I20260809 03:35:36.808300 15072 tablet_replica.cc:330] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d: stopping tablet replica
I20260809 03:35:36.808521 15076 raft_consensus.cc:2229] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:36.808655 15072 raft_consensus.cc:2229] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:36.808586 14265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 03:35:36.808890 15076 raft_consensus.cc:2258] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 03:35:36.809026 14265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:36.809222 15072 raft_consensus.cc:2258] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:36.809427 15077 tablet_replica.cc:330] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7: stopping tablet replica
I20260809 03:35:36.809782 15077 raft_consensus.cc:2229] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:36.810277 15077 raft_consensus.cc:2258] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:36.811929 15077 ts_tablet_manager.cc:1905] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:36.812438 15076 ts_tablet_manager.cc:1905] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:36.814874 15077 ts_tablet_manager.cc:1918] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:36.814986 15077 log.cc:1196] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/wal/wals/8f080f9ffaf94aaf8d356cd7abd7b7de
I20260809 03:35:36.815135 14896 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb94215ec144d789e3f55d833b3158c (DEFAULT_TABLE table=test-table [id=e3baad3a2cdb43c4a84c48bcf3681129]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:36.815318 15077 ts_tablet_manager.cc:1939] T 8f080f9ffaf94aaf8d356cd7abd7b7de P fcacbc2374f245f5b446b989a7c0b1f7: Deleting consensus metadata
I20260809 03:35:36.815444 14896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb94215ec144d789e3f55d833b3158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:36.815495 15076 ts_tablet_manager.cc:1918] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:36.815583 15076 log.cc:1196] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/8f080f9ffaf94aaf8d356cd7abd7b7de
I20260809 03:35:36.815881 15076 ts_tablet_manager.cc:1939] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 21035c20fcd643f8ab869e65744768c9: Deleting consensus metadata
I20260809 03:35:36.817783 15043 tablet_bootstrap.cc:492] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d: Bootstrap starting.
I20260809 03:35:36.818218 14247 catalog_manager.cc:4841] TS 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611): tablet 8f080f9ffaf94aaf8d356cd7abd7b7de (table test-table [id=c593eccda8e34bb59ae9d021757fe77a]) successfully deleted
I20260809 03:35:36.818410 14247 catalog_manager.cc:4841] TS fcacbc2374f245f5b446b989a7c0b1f7 (127.1.224.65:39365): tablet 8f080f9ffaf94aaf8d356cd7abd7b7de (table test-table [id=c593eccda8e34bb59ae9d021757fe77a]) successfully deleted
I20260809 03:35:36.818742 14680 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb94215ec144d789e3f55d833b3158c (DEFAULT_TABLE table=test-table [id=e3baad3a2cdb43c4a84c48bcf3681129]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:36.818843 15043 tablet_bootstrap.cc:654] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:36.819006 14680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb94215ec144d789e3f55d833b3158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:36.819798 14450 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb94215ec144d789e3f55d833b3158c (DEFAULT_TABLE table=test-table [id=e3baad3a2cdb43c4a84c48bcf3681129]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:36.820037 14450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb94215ec144d789e3f55d833b3158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:36.821220 15034 tablet_bootstrap.cc:492] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9: Bootstrap starting.
I20260809 03:35:36.822088 15034 tablet_bootstrap.cc:654] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:36.822088 15072 ts_tablet_manager.cc:1905] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:36.822190 15033 tablet_bootstrap.cc:492] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7: Bootstrap starting.
I20260809 03:35:36.823089 15033 tablet_bootstrap.cc:654] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:36.823791 15034 tablet_bootstrap.cc:492] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9: No bootstrap required, opened a new log
I20260809 03:35:36.823889 15034 ts_tablet_manager.cc:1397] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 03:35:36.824374 15034 raft_consensus.cc:348] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.824559 15034 raft_consensus.cc:374] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:36.824605 15034 raft_consensus.cc:729] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21035c20fcd643f8ab869e65744768c9, State: Initialized, Role: FOLLOWER
I20260809 03:35:36.824724 15033 tablet_bootstrap.cc:492] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7: No bootstrap required, opened a new log
I20260809 03:35:36.824741 15034 consensus_queue.cc:260] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.824813 15033 ts_tablet_manager.cc:1397] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:36.825074 15034 ts_tablet_manager.cc:1428] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 03:35:36.825246 15072 ts_tablet_manager.cc:1918] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:36.825392 15072 log.cc:1196] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/8f080f9ffaf94aaf8d356cd7abd7b7de
I20260809 03:35:36.825583 15033 raft_consensus.cc:348] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.825697 15033 raft_consensus.cc:374] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:36.825733 15033 raft_consensus.cc:729] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcacbc2374f245f5b446b989a7c0b1f7, State: Initialized, Role: FOLLOWER
I20260809 03:35:36.825853 15033 consensus_queue.cc:260] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.825927 15072 ts_tablet_manager.cc:1939] T 8f080f9ffaf94aaf8d356cd7abd7b7de P 2e863fae3a4b4eff91ec29946bd1154d: Deleting consensus metadata
I20260809 03:35:36.826299 15033 ts_tablet_manager.cc:1428] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:36.827594 14244 catalog_manager.cc:4841] TS 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67:41243): tablet 8f080f9ffaf94aaf8d356cd7abd7b7de (table test-table [id=c593eccda8e34bb59ae9d021757fe77a]) successfully deleted
I20260809 03:35:36.828366 15043 tablet_bootstrap.cc:492] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d: No bootstrap required, opened a new log
I20260809 03:35:36.828501 15043 ts_tablet_manager.cc:1397] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.001s
I20260809 03:35:36.828922 15043 raft_consensus.cc:348] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.829042 15043 raft_consensus.cc:374] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:36.829093 15043 raft_consensus.cc:729] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e863fae3a4b4eff91ec29946bd1154d, State: Initialized, Role: FOLLOWER
I20260809 03:35:36.829237 15043 consensus_queue.cc:260] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.829524 15043 ts_tablet_manager.cc:1428] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:36.851199 15042 raft_consensus.cc:482] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:36.851413 15042 raft_consensus.cc:504] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.852993 15042 leader_election.cc:288] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611), 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67:41243)
I20260809 03:35:36.858281 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.853249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:46611 (local address 127.1.224.65:56967)
0809 03:35:36.853674 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.853692 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.853885 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.854301 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.854306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.854321 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:36.854439 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.854446 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.855272 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.855276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.856840 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.856854 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.857798 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.857801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.857891 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.857914 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.858016 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.858095 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":131,"thread_start_us":88,"threads_started":1}
I20260809 03:35:36.858259 15032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.853326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:56967 (local address 127.1.224.66:46611)
0809 03:35:36.853473 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.853478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.853773 (+ 295us) server_negotiation.cc:407] Connection header received
0809 03:35:36.854047 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.854050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.854120 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.854197 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:36.854567 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.855153 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.856998 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.857641 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.857771 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.857831 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.858053 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.858135 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:35:36.858918 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb94215ec144d789e3f55d833b3158c" candidate_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21035c20fcd643f8ab869e65744768c9" is_pre_election: true
I20260809 03:35:36.859102 14714 raft_consensus.cc:2454] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcacbc2374f245f5b446b989a7c0b1f7 in term 0.
I20260809 03:35:36.859220 15029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.853449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:57053 (local address 127.1.224.67:41243)
0809 03:35:36.853569 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.853574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.853907 (+ 333us) server_negotiation.cc:407] Connection header received
0809 03:35:36.853982 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.853986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.854042 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.854132 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:35:36.854484 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.855308 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.857901 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.858507 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.858635 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.858687 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.859011 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.859094 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:35:36.859598 15083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.853255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:41243 (local address 127.1.224.65:57053)
0809 03:35:36.853789 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.853803 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.853886 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.854163 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.854169 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.854191 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:35:36.854350 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.854357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.855426 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.855430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.856840 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.856854 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.858752 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.858756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.858865 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.858887 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.859368 (+ 481us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.859447 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":263,"thread_start_us":121,"threads_started":1}
I20260809 03:35:36.860008 14388 leader_election.cc:302] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21035c20fcd643f8ab869e65744768c9, fcacbc2374f245f5b446b989a7c0b1f7; no voters:
I20260809 03:35:36.860178 14921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb94215ec144d789e3f55d833b3158c" candidate_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e863fae3a4b4eff91ec29946bd1154d" is_pre_election: true
I20260809 03:35:36.860332 15042 raft_consensus.cc:2781] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:36.860394 15042 raft_consensus.cc:482] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:36.860432 15042 raft_consensus.cc:3037] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:36.860464 14921 raft_consensus.cc:2454] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcacbc2374f245f5b446b989a7c0b1f7 in term 0.
I20260809 03:35:36.861791 15042 raft_consensus.cc:504] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.862192 15042 leader_election.cc:288] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [CANDIDATE]: Term 1 election: Requested vote from peers 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611), 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67:41243)
I20260809 03:35:36.862565 14921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb94215ec144d789e3f55d833b3158c" candidate_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e863fae3a4b4eff91ec29946bd1154d"
I20260809 03:35:36.862586 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb94215ec144d789e3f55d833b3158c" candidate_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21035c20fcd643f8ab869e65744768c9"
I20260809 03:35:36.862703 14714 raft_consensus.cc:3037] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:36.862706 14921 raft_consensus.cc:3037] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:36.863863 14921 raft_consensus.cc:2454] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcacbc2374f245f5b446b989a7c0b1f7 in term 1.
I20260809 03:35:36.863883 14714 raft_consensus.cc:2454] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcacbc2374f245f5b446b989a7c0b1f7 in term 1.
I20260809 03:35:36.864192 14386 leader_election.cc:302] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e863fae3a4b4eff91ec29946bd1154d, fcacbc2374f245f5b446b989a7c0b1f7; no voters:
I20260809 03:35:36.864511 15042 raft_consensus.cc:2781] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:36.864758 15042 raft_consensus.cc:686] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 LEADER]: Becoming Leader. State: Replica: fcacbc2374f245f5b446b989a7c0b1f7, State: Running, Role: LEADER
I20260809 03:35:36.865038 15042 consensus_queue.cc:237] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.867812 14265 catalog_manager.cc:5495] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcacbc2374f245f5b446b989a7c0b1f7 (127.1.224.65). New cstate: current_term: 1 leader_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } health_report { overall_health: UNKNOWN } } }
Aug 09 03:35:36 dist-test-slave-jq4t krb5kdc[14201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246535, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.126@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:36.916306 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.906648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:41345 (local address 127.0.0.1:36674)
0809 03:35:36.907017 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:36.907035 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:36.907162 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:36.907459 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:36.907461 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:36.907848 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:36.907972 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.907978 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.909066 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.909069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.909634 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:36.909641 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.910220 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.910223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:36.910333 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.911003 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:36.911015 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:36.912743 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:36.914642 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.914651 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.914655 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.914892 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.915198 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:36.915200 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:36.915201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:36.915286 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:36.915739 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:36.915744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:36.915854 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 03:35:36.915947 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 03:35:36.916095 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":32,"thread_start_us":147,"threads_started":1}
I20260809 03:35:36.916363 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:36.906793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.1.224.126:41345)
0809 03:35:36.906971 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:36.906977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:36.907138 (+ 161us) server_negotiation.cc:407] Connection header received
0809 03:35:36.907256 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:36.907259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:36.907314 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:36.907418 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:36.908093 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.908946 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.909762 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:36.910093 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:36.910207 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:36.912912 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:36.912916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:36.912919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:36.912947 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:36.914527 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.914998 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.915001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.915003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.915099 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:36.915380 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:36.915383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:36.915385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:36.915632 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:36.915739 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:36.916037 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 03:35:36.916167 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":70}
I20260809 03:35:36.920962 14265 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:36674:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 03:35:36.921922 14265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:36.927320 14680 tablet_service.cc:1467] Processing CreateTablet for tablet ae8ba244f71f49d8ac34b350671c25d2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f4605d7ce7ae4ed4a897034fa12e8e9f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:35:36.927361 14896 tablet_service.cc:1467] Processing CreateTablet for tablet ae8ba244f71f49d8ac34b350671c25d2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f4605d7ce7ae4ed4a897034fa12e8e9f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:35:36.927784 14896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8ba244f71f49d8ac34b350671c25d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:36.927791 14680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8ba244f71f49d8ac34b350671c25d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:36.930436 15034 tablet_bootstrap.cc:492] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9: Bootstrap starting.
I20260809 03:35:36.931164 14450 tablet_service.cc:1467] Processing CreateTablet for tablet ae8ba244f71f49d8ac34b350671c25d2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f4605d7ce7ae4ed4a897034fa12e8e9f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:35:36.931679 14450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8ba244f71f49d8ac34b350671c25d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:36.931784 15034 tablet_bootstrap.cc:654] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:36.932739 15043 tablet_bootstrap.cc:492] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d: Bootstrap starting.
I20260809 03:35:36.933830 15043 tablet_bootstrap.cc:654] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:36.935045 15033 tablet_bootstrap.cc:492] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: Bootstrap starting.
I20260809 03:35:36.935382 15034 tablet_bootstrap.cc:492] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9: No bootstrap required, opened a new log
I20260809 03:35:36.935484 15034 ts_tablet_manager.cc:1397] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 03:35:36.935967 15034 raft_consensus.cc:348] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.936097 15033 tablet_bootstrap.cc:654] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:36.936110 15034 raft_consensus.cc:374] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:36.936144 15034 raft_consensus.cc:729] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21035c20fcd643f8ab869e65744768c9, State: Initialized, Role: FOLLOWER
I20260809 03:35:36.936299 15034 consensus_queue.cc:260] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.936620 15034 ts_tablet_manager.cc:1428] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 03:35:36.936640 15045 tablet_replica.cc:439] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } }
I20260809 03:35:36.936811 15045 tablet_replica.cc:442] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:36.937925 15043 tablet_bootstrap.cc:492] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d: No bootstrap required, opened a new log
I20260809 03:35:36.938035 15043 ts_tablet_manager.cc:1397] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 03:35:36.938239 15033 tablet_bootstrap.cc:492] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: No bootstrap required, opened a new log
I20260809 03:35:36.938315 15033 ts_tablet_manager.cc:1397] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:36.938534 15043 raft_consensus.cc:348] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.938644 15043 raft_consensus.cc:374] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:36.938678 15043 raft_consensus.cc:729] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e863fae3a4b4eff91ec29946bd1154d, State: Initialized, Role: FOLLOWER
I20260809 03:35:36.938750 15033 raft_consensus.cc:348] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.938851 15033 raft_consensus.cc:374] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:36.938808 15043 consensus_queue.cc:260] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.938884 15033 raft_consensus.cc:729] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcacbc2374f245f5b446b989a7c0b1f7, State: Initialized, Role: FOLLOWER
I20260809 03:35:36.939018 15033 consensus_queue.cc:260] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:36.939083 15043 ts_tablet_manager.cc:1428] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:36.939282 15033 ts_tablet_manager.cc:1428] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:36.939273 15052 tablet_replica.cc:439] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } }
I20260809 03:35:36.939409 15052 tablet_replica.cc:442] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:36.940436 15042 tablet_replica.cc:439] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } }
I20260809 03:35:36.940616 15042 tablet_replica.cc:442] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 03:35:37.044307 14549 tablet.cc:2369] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:35:37.121912 15042 raft_consensus.cc:482] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:37.122085 15042 raft_consensus.cc:504] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.122519 15042 leader_election.cc:288] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611), 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67:41243)
I20260809 03:35:37.123150 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8ba244f71f49d8ac34b350671c25d2" candidate_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21035c20fcd643f8ab869e65744768c9" is_pre_election: true
I20260809 03:35:37.123150 14921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8ba244f71f49d8ac34b350671c25d2" candidate_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e863fae3a4b4eff91ec29946bd1154d" is_pre_election: true
I20260809 03:35:37.123353 14714 raft_consensus.cc:2454] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcacbc2374f245f5b446b989a7c0b1f7 in term 0.
I20260809 03:35:37.123350 14921 raft_consensus.cc:2454] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcacbc2374f245f5b446b989a7c0b1f7 in term 0.
I20260809 03:35:37.123687 14386 leader_election.cc:302] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e863fae3a4b4eff91ec29946bd1154d, fcacbc2374f245f5b446b989a7c0b1f7; no voters:
I20260809 03:35:37.123914 15042 raft_consensus.cc:2781] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:37.123993 15042 raft_consensus.cc:482] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:37.124054 15042 raft_consensus.cc:3037] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:37.125146 15042 raft_consensus.cc:504] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.125504 15042 leader_election.cc:288] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [CANDIDATE]: Term 1 election: Requested vote from peers 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611), 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67:41243)
I20260809 03:35:37.125922 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8ba244f71f49d8ac34b350671c25d2" candidate_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21035c20fcd643f8ab869e65744768c9"
I20260809 03:35:37.125968 14921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8ba244f71f49d8ac34b350671c25d2" candidate_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e863fae3a4b4eff91ec29946bd1154d"
I20260809 03:35:37.126067 14714 raft_consensus.cc:3037] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:37.126096 14921 raft_consensus.cc:3037] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:37.127014 14714 raft_consensus.cc:2454] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcacbc2374f245f5b446b989a7c0b1f7 in term 1.
I20260809 03:35:37.127100 14921 raft_consensus.cc:2454] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcacbc2374f245f5b446b989a7c0b1f7 in term 1.
I20260809 03:35:37.127380 14388 leader_election.cc:302] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21035c20fcd643f8ab869e65744768c9, fcacbc2374f245f5b446b989a7c0b1f7; no voters:
I20260809 03:35:37.127552 15042 raft_consensus.cc:2781] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:37.127705 15042 raft_consensus.cc:686] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 LEADER]: Becoming Leader. State: Replica: fcacbc2374f245f5b446b989a7c0b1f7, State: Running, Role: LEADER
I20260809 03:35:37.127859 15042 consensus_queue.cc:237] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.128681 15085 tablet_replica.cc:439] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: TxnStatusTablet state changed. Reason: New leader fcacbc2374f245f5b446b989a7c0b1f7. Latest consensus state: current_term: 1 leader_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } }
I20260809 03:35:37.128825 15085 tablet_replica.cc:442] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:35:37.129151 15102 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 03:35:37.129379 15102 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 03:35:37.129931 14263 catalog_manager.cc:5495] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcacbc2374f245f5b446b989a7c0b1f7 (127.1.224.65). New cstate: current_term: 1 leader_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260809 03:35:37.180649 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:37.176401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39365 (local address 127.0.0.1:36260)
0809 03:35:37.176576 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:37.176592 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:37.176705 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:37.177062 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:37.177064 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:37.177070 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:35:37.177171 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:37.177175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.178207 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.178210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:37.178818 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:37.178826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.179402 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.179405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:37.179474 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:37.179523 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:35:37.180324 (+ 801us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:35:37.180338 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 03:35:37.180389 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 03:35:37.180468 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 03:35:37.180653 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:37.176548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36260 (local address 127.1.224.65:39365)
0809 03:35:37.176683 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:37.176688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:37.176705 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:35:37.176857 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:37.176861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:37.176936 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:37.177044 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:37.177290 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.178090 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.178935 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.179277 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.179416 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:37.179669 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:37.180226 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:37.180345 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:37.180419 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 03:35:37.180509 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:35:37.184101 14714 raft_consensus.cc:1261] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Refusing update from remote peer fcacbc2374f245f5b446b989a7c0b1f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:37.184303 14921 raft_consensus.cc:1261] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Refusing update from remote peer fcacbc2374f245f5b446b989a7c0b1f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:37.184762 15085 consensus_queue.cc:1035] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:37.184937 15042 consensus_queue.cc:1035] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:37.186902 15045 tablet_replica.cc:439] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9: TxnStatusTablet state changed. Reason: New leader fcacbc2374f245f5b446b989a7c0b1f7. Latest consensus state: current_term: 1 leader_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } }
I20260809 03:35:37.187031 15045 tablet_replica.cc:442] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:37.186964 15052 tablet_replica.cc:439] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d: TxnStatusTablet state changed. Reason: New leader fcacbc2374f245f5b446b989a7c0b1f7. Latest consensus state: current_term: 1 leader_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } }
I20260809 03:35:37.187073 15052 tablet_replica.cc:442] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:37.189235 15042 tablet_replica.cc:439] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } }
I20260809 03:35:37.189359 15042 tablet_replica.cc:442] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:35:37.190717 15045 tablet_replica.cc:439] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } }
I20260809 03:35:37.190847 15045 tablet_replica.cc:442] T ae8ba244f71f49d8ac34b350671c25d2 P 21035c20fcd643f8ab869e65744768c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:37.192850 15085 tablet_replica.cc:439] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } }
I20260809 03:35:37.192999 15085 tablet_replica.cc:442] T ae8ba244f71f49d8ac34b350671c25d2 P fcacbc2374f245f5b446b989a7c0b1f7: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:35:37.193539 15052 tablet_replica.cc:439] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } } }
I20260809 03:35:37.193666 15052 tablet_replica.cc:442] T ae8ba244f71f49d8ac34b350671c25d2 P 2e863fae3a4b4eff91ec29946bd1154d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:35:37.198522 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:37.194700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.1.224.65:39365)
0809 03:35:37.194847 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:37.194852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:37.194866 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:35:37.195027 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:37.195030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:37.195079 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:37.195170 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:37.195462 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.196233 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.197488 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.197705 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.197781 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:37.198077 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:37.198145 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:37.198195 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:37.198316 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 03:35:37.198362 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 03:35:37.209596 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:37.206191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:56013 (local address 127.1.224.65:39365)
0809 03:35:37.206320 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:37.206336 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:37.206510 (+ 174us) server_negotiation.cc:407] Connection header received
0809 03:35:37.206671 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:37.206675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:37.206724 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:37.206806 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:37.207092 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.207590 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.208439 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.208708 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.208820 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:37.209126 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:37.209205 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:37.209287 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:37.209379 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 03:35:37.209431 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":46}
I20260809 03:35:37.209614 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:37.206072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39365 (local address 127.1.224.65:56013)
0809 03:35:37.206423 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:37.206436 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:37.206563 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:37.206815 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:37.206818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:37.206824 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:35:37.206981 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:37.206986 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.207701 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.207705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:37.208319 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:37.208327 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.208820 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.208823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:37.208948 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:37.208990 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:35:37.209302 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:35:37.209309 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 03:35:37.209370 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 03:35:37.209449 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":120,"threads_started":1}
I20260809 03:35:37.216428 14921 raft_consensus.cc:1261] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Refusing update from remote peer fcacbc2374f245f5b446b989a7c0b1f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:37.216914 15111 consensus_queue.cc:1035] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:37.216997 14714 raft_consensus.cc:1261] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Refusing update from remote peer fcacbc2374f245f5b446b989a7c0b1f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:37.217332 15111 consensus_queue.cc:1035] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:37.223014 15057 mvcc.cc:205] Tried to move back new op lower bound from 7316465816435060736 to 7316465815002046464. Current Snapshot: MvccSnapshot[applied={T|T < 7316465816435060736}]
I20260809 03:35:37.259761 14265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:37.259950 14265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:37.262202 14265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1: Sending DeleteTablet for 3 replicas of tablet 7cb94215ec144d789e3f55d833b3158c
I20260809 03:35:37.262813 14449 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb94215ec144d789e3f55d833b3158c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:37 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:36256
I20260809 03:35:37.262866 14680 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb94215ec144d789e3f55d833b3158c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:37 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:51952
I20260809 03:35:37.262861 14896 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb94215ec144d789e3f55d833b3158c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:37 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:53986
I20260809 03:35:37.263017 15077 tablet_replica.cc:330] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7: stopping tablet replica
I20260809 03:35:37.263058 15076 tablet_replica.cc:330] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9: stopping tablet replica
I20260809 03:35:37.263058 15072 tablet_replica.cc:330] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d: stopping tablet replica
I20260809 03:35:37.263132 15077 raft_consensus.cc:2229] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:37.263161 15076 raft_consensus.cc:2229] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:37.263162 15072 raft_consensus.cc:2229] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:37.263274 15076 raft_consensus.cc:2258] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:37.263275 15072 raft_consensus.cc:2258] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:37.263447 15077 raft_consensus.cc:2258] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:37.264252 15072 ts_tablet_manager.cc:1905] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:37.264266 15077 ts_tablet_manager.cc:1905] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:37.264652 15076 ts_tablet_manager.cc:1905] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:37.267535 15077 ts_tablet_manager.cc:1918] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:37.267633 15077 log.cc:1196] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/wal/wals/7cb94215ec144d789e3f55d833b3158c
I20260809 03:35:37.267930 15077 ts_tablet_manager.cc:1939] T 7cb94215ec144d789e3f55d833b3158c P fcacbc2374f245f5b446b989a7c0b1f7: Deleting consensus metadata
I20260809 03:35:37.268123 15076 ts_tablet_manager.cc:1918] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:37.268280 15076 log.cc:1196] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/7cb94215ec144d789e3f55d833b3158c
I20260809 03:35:37.268636 15076 ts_tablet_manager.cc:1939] T 7cb94215ec144d789e3f55d833b3158c P 21035c20fcd643f8ab869e65744768c9: Deleting consensus metadata
I20260809 03:35:37.268800 14247 catalog_manager.cc:4841] TS fcacbc2374f245f5b446b989a7c0b1f7 (127.1.224.65:39365): tablet 7cb94215ec144d789e3f55d833b3158c (table test-table [id=e3baad3a2cdb43c4a84c48bcf3681129]) successfully deleted
I20260809 03:35:37.269784 14247 catalog_manager.cc:4841] TS 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611): tablet 7cb94215ec144d789e3f55d833b3158c (table test-table [id=e3baad3a2cdb43c4a84c48bcf3681129]) successfully deleted
I20260809 03:35:37.272580 15072 ts_tablet_manager.cc:1918] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:37.272679 15072 log.cc:1196] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/7cb94215ec144d789e3f55d833b3158c
I20260809 03:35:37.272951 15072 ts_tablet_manager.cc:1939] T 7cb94215ec144d789e3f55d833b3158c P 2e863fae3a4b4eff91ec29946bd1154d: Deleting consensus metadata
I20260809 03:35:37.273794 14244 catalog_manager.cc:4841] TS 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67:41243): tablet 7cb94215ec144d789e3f55d833b3158c (table test-table [id=e3baad3a2cdb43c4a84c48bcf3681129]) successfully deleted
Aug 09 03:35:37 dist-test-slave-jq4t krb5kdc[14201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:35:37 dist-test-slave-jq4t krb5kdc[14201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:37 dist-test-slave-jq4t krb5kdc[14201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:35:37.301007 14330 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:37.292512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36686 (local address 127.1.224.126:41345)
0809 03:35:37.292654 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:37.292659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:37.293175 (+ 516us) server_negotiation.cc:407] Connection header received
0809 03:35:37.293403 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:37.293407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:37.293468 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:37.293563 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:37.294524 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.295421 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.296697 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.297037 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.297145 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:37.300862 (+ 3717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36686: BlockingRecv error: recv got EOF from 127.0.0.1:36686 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 03:35:37 dist-test-slave-jq4t krb5kdc[14201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM
I20260809 03:35:37.315161 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:37.304899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.1.224.126:41345)
0809 03:35:37.305016 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:37.305020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:37.305269 (+ 249us) server_negotiation.cc:407] Connection header received
0809 03:35:37.305371 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:37.305373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:37.305424 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:37.305504 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:37.306238 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.307021 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.308145 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.308526 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.308632 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:37.311322 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:37.311327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:37.311330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:37.311369 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:37.313243 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:37.313790 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:37.313794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:37.313796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:37.313895 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:37.314219 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:37.314222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:37.314224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:37.314480 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:37.314614 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:37.314851 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 03:35:37.314984 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:35:37.318413 14265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:35:37.318918 14265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:37.324173 14680 tablet_service.cc:1467] Processing CreateTablet for tablet f5205fb191454b768ee98a18f3a3be4a (DEFAULT_TABLE table=test-table [id=f748a3570e3847b189c80b0f133dc539]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:37.324185 14449 tablet_service.cc:1467] Processing CreateTablet for tablet f5205fb191454b768ee98a18f3a3be4a (DEFAULT_TABLE table=test-table [id=f748a3570e3847b189c80b0f133dc539]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:37.324587 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5205fb191454b768ee98a18f3a3be4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:37.324604 14680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5205fb191454b768ee98a18f3a3be4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:37.326738 15033 tablet_bootstrap.cc:492] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7: Bootstrap starting.
I20260809 03:35:37.327025 14896 tablet_service.cc:1467] Processing CreateTablet for tablet f5205fb191454b768ee98a18f3a3be4a (DEFAULT_TABLE table=test-table [id=f748a3570e3847b189c80b0f133dc539]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:37.327325 14896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5205fb191454b768ee98a18f3a3be4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:37.327678 15033 tablet_bootstrap.cc:654] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:37.328675 15034 tablet_bootstrap.cc:492] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9: Bootstrap starting.
I20260809 03:35:37.329574 15034 tablet_bootstrap.cc:654] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:37.329612 15043 tablet_bootstrap.cc:492] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d: Bootstrap starting.
I20260809 03:35:37.330509 15043 tablet_bootstrap.cc:654] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:37.330904 15034 tablet_bootstrap.cc:492] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9: No bootstrap required, opened a new log
I20260809 03:35:37.331003 15034 ts_tablet_manager.cc:1397] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:37.331558 15033 tablet_bootstrap.cc:492] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7: No bootstrap required, opened a new log
I20260809 03:35:37.331660 15033 ts_tablet_manager.cc:1397] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 03:35:37.331844 15034 raft_consensus.cc:348] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.331991 15034 raft_consensus.cc:374] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:37.332036 15034 raft_consensus.cc:729] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21035c20fcd643f8ab869e65744768c9, State: Initialized, Role: FOLLOWER
I20260809 03:35:37.332108 15033 raft_consensus.cc:348] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.332227 15033 raft_consensus.cc:374] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:37.332176 15034 consensus_queue.cc:260] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.332266 15033 raft_consensus.cc:729] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcacbc2374f245f5b446b989a7c0b1f7, State: Initialized, Role: FOLLOWER
I20260809 03:35:37.332396 15033 consensus_queue.cc:260] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.332460 15034 ts_tablet_manager.cc:1428] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:35:37.332877 15033 ts_tablet_manager.cc:1428] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 03:35:37.333977 15043 tablet_bootstrap.cc:492] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d: No bootstrap required, opened a new log
I20260809 03:35:37.334095 15043 ts_tablet_manager.cc:1397] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 03:35:37.334535 15043 raft_consensus.cc:348] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.334656 15043 raft_consensus.cc:374] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:37.334694 15043 raft_consensus.cc:729] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e863fae3a4b4eff91ec29946bd1154d, State: Initialized, Role: FOLLOWER
I20260809 03:35:37.334862 15043 consensus_queue.cc:260] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.335158 15043 ts_tablet_manager.cc:1428] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:37.372701 15052 raft_consensus.cc:482] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:37.372879 15052 raft_consensus.cc:504] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.373329 15052 leader_election.cc:288] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcacbc2374f245f5b446b989a7c0b1f7 (127.1.224.65:39365), 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611)
I20260809 03:35:37.373840 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5205fb191454b768ee98a18f3a3be4a" candidate_uuid: "2e863fae3a4b4eff91ec29946bd1154d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21035c20fcd643f8ab869e65744768c9" is_pre_election: true
I20260809 03:35:37.373797 14477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5205fb191454b768ee98a18f3a3be4a" candidate_uuid: "2e863fae3a4b4eff91ec29946bd1154d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" is_pre_election: true
I20260809 03:35:37.374014 14714 raft_consensus.cc:2454] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e863fae3a4b4eff91ec29946bd1154d in term 0.
I20260809 03:35:37.374014 14477 raft_consensus.cc:2454] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e863fae3a4b4eff91ec29946bd1154d in term 0.
I20260809 03:35:37.374316 14823 leader_election.cc:302] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e863fae3a4b4eff91ec29946bd1154d, fcacbc2374f245f5b446b989a7c0b1f7; no voters:
I20260809 03:35:37.374541 15052 raft_consensus.cc:2781] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:37.374616 15052 raft_consensus.cc:482] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:37.374655 15052 raft_consensus.cc:3037] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:37.375979 15052 raft_consensus.cc:504] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.376338 15052 leader_election.cc:288] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [CANDIDATE]: Term 1 election: Requested vote from peers fcacbc2374f245f5b446b989a7c0b1f7 (127.1.224.65:39365), 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611)
I20260809 03:35:37.376806 14477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5205fb191454b768ee98a18f3a3be4a" candidate_uuid: "2e863fae3a4b4eff91ec29946bd1154d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcacbc2374f245f5b446b989a7c0b1f7"
I20260809 03:35:37.376816 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5205fb191454b768ee98a18f3a3be4a" candidate_uuid: "2e863fae3a4b4eff91ec29946bd1154d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21035c20fcd643f8ab869e65744768c9"
I20260809 03:35:37.376945 14714 raft_consensus.cc:3037] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:37.376974 14477 raft_consensus.cc:3037] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:37.378073 14477 raft_consensus.cc:2454] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e863fae3a4b4eff91ec29946bd1154d in term 1.
I20260809 03:35:37.378083 14714 raft_consensus.cc:2454] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e863fae3a4b4eff91ec29946bd1154d in term 1.
I20260809 03:35:37.378422 14823 leader_election.cc:302] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21035c20fcd643f8ab869e65744768c9, 2e863fae3a4b4eff91ec29946bd1154d; no voters:
I20260809 03:35:37.378688 15052 raft_consensus.cc:2781] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:37.378865 15052 raft_consensus.cc:686] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 1 LEADER]: Becoming Leader. State: Replica: 2e863fae3a4b4eff91ec29946bd1154d, State: Running, Role: LEADER
I20260809 03:35:37.379014 15052 consensus_queue.cc:237] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.380798 14267 catalog_manager.cc:5495] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67). New cstate: current_term: 1 leader_uuid: "2e863fae3a4b4eff91ec29946bd1154d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } health_report { overall_health: HEALTHY } } }
I20260809 03:35:37.391913 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:37.388120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53992 (local address 127.1.224.67:41243)
0809 03:35:37.388361 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:37.388365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:37.388396 (+ 31us) server_negotiation.cc:407] Connection header received
0809 03:35:37.388456 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:37.388459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:37.388565 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:37.388649 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:37.388924 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.389539 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.390777 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.391017 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.391095 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:37.391399 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:37.391481 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:37.391532 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:37.391650 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 03:35:37.391736 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":89,"threads_started":1}
I20260809 03:35:37.394048 14477 raft_consensus.cc:1261] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Refusing update from remote peer 2e863fae3a4b4eff91ec29946bd1154d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:37.394052 14714 raft_consensus.cc:1261] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Refusing update from remote peer 2e863fae3a4b4eff91ec29946bd1154d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:37.394574 15151 consensus_queue.cc:1035] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:37.394734 15052 consensus_queue.cc:1035] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [LEADER]: Connected to new peer: Peer: permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:37.404325 14267 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:37.404531 14267 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:37.406674 14267 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1: Sending DeleteTablet for 3 replicas of tablet f5205fb191454b768ee98a18f3a3be4a
I20260809 03:35:37.407300 14680 tablet_service.cc:1514] Processing DeleteTablet for tablet f5205fb191454b768ee98a18f3a3be4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:37 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:51952
I20260809 03:35:37.407418 14896 tablet_service.cc:1514] Processing DeleteTablet for tablet f5205fb191454b768ee98a18f3a3be4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:37 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:53986
I20260809 03:35:37.407514 15076 tablet_replica.cc:330] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9: stopping tablet replica
I20260809 03:35:37.407596 15072 tablet_replica.cc:330] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d: stopping tablet replica
I20260809 03:35:37.407624 15076 raft_consensus.cc:2229] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:37.407717 15072 raft_consensus.cc:2229] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:37.407734 15076 raft_consensus.cc:2258] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:37.407925 15072 raft_consensus.cc:2258] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:37.408221 14449 tablet_service.cc:1514] Processing DeleteTablet for tablet f5205fb191454b768ee98a18f3a3be4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:37 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:36256
I20260809 03:35:37.408533 15077 tablet_replica.cc:330] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7: stopping tablet replica
I20260809 03:35:37.408525 14267 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 03:35:37.408700 15077 raft_consensus.cc:2229] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 03:35:37.409036 14267 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:35:37.408969 15077 raft_consensus.cc:2258] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:37.409914 15077 ts_tablet_manager.cc:1905] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:37.410627 15076 ts_tablet_manager.cc:1905] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:37.412530 15077 ts_tablet_manager.cc:1918] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:37.412607 15077 log.cc:1196] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-0/wal/wals/f5205fb191454b768ee98a18f3a3be4a
I20260809 03:35:37.412767 15076 ts_tablet_manager.cc:1918] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:37.412848 15076 log.cc:1196] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/f5205fb191454b768ee98a18f3a3be4a
I20260809 03:35:37.412865 15077 ts_tablet_manager.cc:1939] T f5205fb191454b768ee98a18f3a3be4a P fcacbc2374f245f5b446b989a7c0b1f7: Deleting consensus metadata
I20260809 03:35:37.413136 15076 ts_tablet_manager.cc:1939] T f5205fb191454b768ee98a18f3a3be4a P 21035c20fcd643f8ab869e65744768c9: Deleting consensus metadata
I20260809 03:35:37.413926 14247 catalog_manager.cc:4841] TS 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611): tablet f5205fb191454b768ee98a18f3a3be4a (table test-table [id=f748a3570e3847b189c80b0f133dc539]) successfully deleted
I20260809 03:35:37.414100 15072 ts_tablet_manager.cc:1905] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:37.414574 14247 catalog_manager.cc:4841] TS fcacbc2374f245f5b446b989a7c0b1f7 (127.1.224.65:39365): tablet f5205fb191454b768ee98a18f3a3be4a (table test-table [id=f748a3570e3847b189c80b0f133dc539]) successfully deleted
I20260809 03:35:37.416240 15072 ts_tablet_manager.cc:1918] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:35:37.416309 15072 log.cc:1196] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/f5205fb191454b768ee98a18f3a3be4a
I20260809 03:35:37.416644 15072 ts_tablet_manager.cc:1939] T f5205fb191454b768ee98a18f3a3be4a P 2e863fae3a4b4eff91ec29946bd1154d: Deleting consensus metadata
I20260809 03:35:37.417426 14244 catalog_manager.cc:4841] TS 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67:41243): tablet f5205fb191454b768ee98a18f3a3be4a (table test-table [id=f748a3570e3847b189c80b0f133dc539]) successfully deleted
I20260809 03:35:37.420257 14449 tablet_service.cc:1467] Processing CreateTablet for tablet 05486ba5e1d547de8b50f44eed83a8e1 (DEFAULT_TABLE table=test-table [id=7be7c01a15904364a93d20f23c9547c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:37.420347 14896 tablet_service.cc:1467] Processing CreateTablet for tablet 05486ba5e1d547de8b50f44eed83a8e1 (DEFAULT_TABLE table=test-table [id=7be7c01a15904364a93d20f23c9547c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:37.420591 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05486ba5e1d547de8b50f44eed83a8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:37.420661 14896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05486ba5e1d547de8b50f44eed83a8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:37.422556 15043 tablet_bootstrap.cc:492] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d: Bootstrap starting.
I20260809 03:35:37.422787 14680 tablet_service.cc:1467] Processing CreateTablet for tablet 05486ba5e1d547de8b50f44eed83a8e1 (DEFAULT_TABLE table=test-table [id=7be7c01a15904364a93d20f23c9547c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:35:37.423086 14680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05486ba5e1d547de8b50f44eed83a8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:37.423470 15033 tablet_bootstrap.cc:492] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7: Bootstrap starting.
I20260809 03:35:37.423575 15043 tablet_bootstrap.cc:654] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:37.424244 15033 tablet_bootstrap.cc:654] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:37.425303 15043 tablet_bootstrap.cc:492] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d: No bootstrap required, opened a new log
I20260809 03:35:37.425397 15043 ts_tablet_manager.cc:1397] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 03:35:37.425675 15034 tablet_bootstrap.cc:492] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9: Bootstrap starting.
I20260809 03:35:37.425832 15043 raft_consensus.cc:348] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.425937 15043 raft_consensus.cc:374] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:37.425973 15043 raft_consensus.cc:729] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e863fae3a4b4eff91ec29946bd1154d, State: Initialized, Role: FOLLOWER
I20260809 03:35:37.426110 15043 consensus_queue.cc:260] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.426307 15033 tablet_bootstrap.cc:492] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7: No bootstrap required, opened a new log
I20260809 03:35:37.426383 15043 ts_tablet_manager.cc:1428] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:37.426400 15033 ts_tablet_manager.cc:1397] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 03:35:37.426625 15034 tablet_bootstrap.cc:654] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:37.426792 15033 raft_consensus.cc:348] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.426911 15033 raft_consensus.cc:374] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:37.426942 15033 raft_consensus.cc:729] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcacbc2374f245f5b446b989a7c0b1f7, State: Initialized, Role: FOLLOWER
I20260809 03:35:37.427062 15033 consensus_queue.cc:260] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.427348 15033 ts_tablet_manager.cc:1428] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:35:37.428046 15034 tablet_bootstrap.cc:492] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9: No bootstrap required, opened a new log
I20260809 03:35:37.428158 15034 ts_tablet_manager.cc:1397] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 03:35:37.428648 15034 raft_consensus.cc:348] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.428769 15034 raft_consensus.cc:374] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:37.428805 15034 raft_consensus.cc:729] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21035c20fcd643f8ab869e65744768c9, State: Initialized, Role: FOLLOWER
I20260809 03:35:37.428920 15034 consensus_queue.cc:260] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.429206 15034 ts_tablet_manager.cc:1428] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 03:35:37.490496 15085 raft_consensus.cc:482] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:35:37.490715 15085 raft_consensus.cc:504] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.491264 15085 leader_election.cc:288] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611), 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67:41243)
I20260809 03:35:37.491820 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05486ba5e1d547de8b50f44eed83a8e1" candidate_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21035c20fcd643f8ab869e65744768c9" is_pre_election: true
I20260809 03:35:37.491881 14920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05486ba5e1d547de8b50f44eed83a8e1" candidate_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e863fae3a4b4eff91ec29946bd1154d" is_pre_election: true
I20260809 03:35:37.492044 14714 raft_consensus.cc:2454] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcacbc2374f245f5b446b989a7c0b1f7 in term 0.
I20260809 03:35:37.492079 14920 raft_consensus.cc:2454] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcacbc2374f245f5b446b989a7c0b1f7 in term 0.
I20260809 03:35:37.492376 14388 leader_election.cc:302] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21035c20fcd643f8ab869e65744768c9, fcacbc2374f245f5b446b989a7c0b1f7; no voters:
I20260809 03:35:37.492595 15085 raft_consensus.cc:2781] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:35:37.492663 15085 raft_consensus.cc:482] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:35:37.492700 15085 raft_consensus.cc:3037] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:37.493878 15085 raft_consensus.cc:504] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.494269 15085 leader_election.cc:288] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [CANDIDATE]: Term 1 election: Requested vote from peers 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611), 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67:41243)
I20260809 03:35:37.494661 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05486ba5e1d547de8b50f44eed83a8e1" candidate_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21035c20fcd643f8ab869e65744768c9"
I20260809 03:35:37.494659 14920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05486ba5e1d547de8b50f44eed83a8e1" candidate_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e863fae3a4b4eff91ec29946bd1154d"
I20260809 03:35:37.494797 14714 raft_consensus.cc:3037] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:37.494812 14920 raft_consensus.cc:3037] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:37.495741 14714 raft_consensus.cc:2454] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcacbc2374f245f5b446b989a7c0b1f7 in term 1.
I20260809 03:35:37.495780 14920 raft_consensus.cc:2454] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcacbc2374f245f5b446b989a7c0b1f7 in term 1.
I20260809 03:35:37.496055 14388 leader_election.cc:302] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21035c20fcd643f8ab869e65744768c9, fcacbc2374f245f5b446b989a7c0b1f7; no voters:
I20260809 03:35:37.496294 15085 raft_consensus.cc:2781] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:37.496417 15085 raft_consensus.cc:686] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 LEADER]: Becoming Leader. State: Replica: fcacbc2374f245f5b446b989a7c0b1f7, State: Running, Role: LEADER
I20260809 03:35:37.496606 15085 consensus_queue.cc:237] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } }
I20260809 03:35:37.498584 14267 catalog_manager.cc:5495] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcacbc2374f245f5b446b989a7c0b1f7 (127.1.224.65). New cstate: current_term: 1 leader_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcacbc2374f245f5b446b989a7c0b1f7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 } health_report { overall_health: UNKNOWN } } }
I20260809 03:35:37.520222 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:37.517073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36288 (local address 127.1.224.65:39365)
0809 03:35:37.517214 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:37.517218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:37.517229 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:35:37.517267 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:37.517269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:37.517353 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:37.517429 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:35:37.517722 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.518253 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.519113 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:37.519356 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:37.519491 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:37.519742 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:35:37.519830 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:35:37.519918 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:37.520019 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 03:35:37.520074 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:35:37.526881 14711 raft_consensus.cc:1261] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Refusing update from remote peer fcacbc2374f245f5b446b989a7c0b1f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:37.527321 15111 consensus_queue.cc:1035] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21035c20fcd643f8ab869e65744768c9" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:37.527418 14920 raft_consensus.cc:1261] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Refusing update from remote peer fcacbc2374f245f5b446b989a7c0b1f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:35:37.527798 15111 consensus_queue.cc:1035] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e863fae3a4b4eff91ec29946bd1154d" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 41243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:35:37.565611 14267 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:37.565778 14267 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:35:37.568025 14267 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4c0475a604794988a8e8514cdae680f1: Sending DeleteTablet for 3 replicas of tablet 05486ba5e1d547de8b50f44eed83a8e1
I20260809 03:35:37.568703 14450 tablet_service.cc:1514] Processing DeleteTablet for tablet 05486ba5e1d547de8b50f44eed83a8e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:37 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:36256
I20260809 03:35:37.568813 14896 tablet_service.cc:1514] Processing DeleteTablet for tablet 05486ba5e1d547de8b50f44eed83a8e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:37 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:53986
I20260809 03:35:37.568943 14680 tablet_service.cc:1514] Processing DeleteTablet for tablet 05486ba5e1d547de8b50f44eed83a8e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:35:37 UTC) from {username='oryx', principal='oryx/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:51952
I20260809 03:35:37.569018 15072 tablet_replica.cc:330] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d: stopping tablet replica
I20260809 03:35:37.569132 15077 tablet_replica.cc:330] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7: stopping tablet replica
I20260809 03:35:37.569150 15072 raft_consensus.cc:2229] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:37.569249 15077 raft_consensus.cc:2229] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:35:37.569280 15072 raft_consensus.cc:2258] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:37.569398 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 14366
I20260809 03:35:37.569478 15077 raft_consensus.cc:2258] T 05486ba5e1d547de8b50f44eed83a8e1 P fcacbc2374f245f5b446b989a7c0b1f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:37.569681 15076 tablet_replica.cc:330] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9: stopping tablet replica
I20260809 03:35:37.569824 15076 raft_consensus.cc:2229] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:35:37.569973 15076 raft_consensus.cc:2258] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:35:37.577607 15072 ts_tablet_manager.cc:1905] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:37.579540 15076 ts_tablet_manager.cc:1905] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:35:37.579991 15072 ts_tablet_manager.cc:1918] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:37.580054 15072 log.cc:1196] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/05486ba5e1d547de8b50f44eed83a8e1
I20260809 03:35:37.580250 15072 ts_tablet_manager.cc:1939] T 05486ba5e1d547de8b50f44eed83a8e1 P 2e863fae3a4b4eff91ec29946bd1154d: Deleting consensus metadata
I20260809 03:35:37.581066 14244 catalog_manager.cc:4841] TS 2e863fae3a4b4eff91ec29946bd1154d (127.1.224.67:41243): tablet 05486ba5e1d547de8b50f44eed83a8e1 (table test-table [id=7be7c01a15904364a93d20f23c9547c0]) successfully deleted
I20260809 03:35:37.582859 15076 ts_tablet_manager.cc:1918] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:35:37.582953 15076 log.cc:1196] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/05486ba5e1d547de8b50f44eed83a8e1
I20260809 03:35:37.583213 15076 ts_tablet_manager.cc:1939] T 05486ba5e1d547de8b50f44eed83a8e1 P 21035c20fcd643f8ab869e65744768c9: Deleting consensus metadata
I20260809 03:35:37.584056 14247 catalog_manager.cc:4841] TS 21035c20fcd643f8ab869e65744768c9 (127.1.224.66:46611): tablet 05486ba5e1d547de8b50f44eed83a8e1 (table test-table [id=7be7c01a15904364a93d20f23c9547c0]) successfully deleted
I20260809 03:35:37.588269 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 14564
W20260809 03:35:37.588212 14247 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:39365 (error 108)
I20260809 03:35:37.596177 15173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:37.594702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39365 (local address 127.0.0.1:36292)
0809 03:35:37.596002 (+ 1300us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:37.596075 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:39365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1223,"thread_start_us":72,"threads_started":1}
W20260809 03:35:37.596354 14247 catalog_manager.cc:4584] TS fcacbc2374f245f5b446b989a7c0b1f7 (127.1.224.65:39365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05486ba5e1d547de8b50f44eed83a8e1: Network error: Client connection negotiation failed: client connection to 127.1.224.65:39365: connect: Connection refused (error 111)
I20260809 03:35:37.602345 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 14791
I20260809 03:35:37.627815 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 14219
2026-08-09T03:35:37Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3499 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:37 dist-test-slave-jq4t krb5kdc[15178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:37 dist-test-slave-jq4t krb5kdc[15178](info): set up 2 sockets
Aug 09 03:35:37 dist-test-slave-jq4t krb5kdc[15178](info): commencing operation
krb5kdc: starting...
W20260809 03:35:38.887681 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:35:38 dist-test-slave-jq4t krb5kdc[15178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:35:39.019359 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:42981
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:34393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:42981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:39.139123 15196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:39.139492 15196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:39.139582 15196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:39.143833 15196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:39.143951 15196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:39.143988 15196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:39.144027 15196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:39.144057 15196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:39.149734 15196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:34393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:42981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:42981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:39.150980 15196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:39.152096 15196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:39.161032 15204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:39.161185 15196 server_base.cc:1034] running on GCE node
W20260809 03:35:39.161059 15203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:39.161362 15207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:39.161986 15196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:39.163201 15196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:39.164405 15196 hybrid_clock.cc:648] HybridClock initialized: now 1786246539164385 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T03:35:39Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1542 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:39 dist-test-slave-jq4t krb5kdc[15213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:39 dist-test-slave-jq4t krb5kdc[15213](info): set up 2 sockets
Aug 09 03:35:39 dist-test-slave-jq4t krb5kdc[15213](info): commencing operation
krb5kdc: starting...
W20260809 03:35:40.426833 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:35:40 dist-test-slave-jq4t krb5kdc[15213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:35:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:35:40.561857 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:42451
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:42451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:40.679440 15230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:40.679728 15230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:40.679769 15230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:40.683894 15230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:40.683972 15230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:40.683992 15230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:40.684010 15230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:40.684026 15230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:40.689939 15230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:42451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:42451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:40.691329 15230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:40.692456 15230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:40.701944 15240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:40.701944 15244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:40.701946 15237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:40.702289 15230 server_base.cc:1034] running on GCE node
I20260809 03:35:40.703066 15230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:40.704385 15230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:40.705627 15230 hybrid_clock.cc:648] HybridClock initialized: now 1786246540705602 us; error 42 us; skew 500 ppm
Aug 09 03:35:40 dist-test-slave-jq4t krb5kdc[15213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246540, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:40.713403 15230 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:35:40.714608 15230 webserver.cc:460] Webserver started at http://127.1.224.126:42669/ using document root <none> and password file <none>
I20260809 03:35:40.715296 15230 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:40.715399 15230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:40.715663 15230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:40.718050 15230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "ce3dd89e74c540c791b8d7dd7972ad84"
format_stamp: "Formatted at 2026-08-09 03:35:40 on dist-test-slave-jq4t"
server_key: "266e170d3076ca43a597d077ada002d1"
server_key_iv: "73295f2eb1ad9207e4587b570d51d721"
server_key_version: "encryptionkey@0"
I20260809 03:35:40.718664 15230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "ce3dd89e74c540c791b8d7dd7972ad84"
format_stamp: "Formatted at 2026-08-09 03:35:40 on dist-test-slave-jq4t"
server_key: "266e170d3076ca43a597d077ada002d1"
server_key_iv: "73295f2eb1ad9207e4587b570d51d721"
server_key_version: "encryptionkey@0"
I20260809 03:35:40.722818 15230 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:40.725513 15253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:40.726750 15230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:40.726920 15230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "ce3dd89e74c540c791b8d7dd7972ad84"
format_stamp: "Formatted at 2026-08-09 03:35:40 on dist-test-slave-jq4t"
server_key: "266e170d3076ca43a597d077ada002d1"
server_key_iv: "73295f2eb1ad9207e4587b570d51d721"
server_key_version: "encryptionkey@0"
I20260809 03:35:40.727044 15230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:40.738154 15230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:40.741586 15230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:40.741819 15230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:40.749413 15230 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:42451
I20260809 03:35:40.749439 15338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:42451 every 8 connection(s)
I20260809 03:35:40.750447 15230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:40.753374 15340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:40.759702 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 15230
I20260809 03:35:40.759739 15340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: Bootstrap starting.
I20260809 03:35:40.759817 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:40.760128 1921 external_mini_cluster.cc:1428] Setting key 0c443d271a5ce0698fbdfa5d878a28fb
I20260809 03:35:40.762778 15340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:40.763757 15340 log.cc:824] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:40.766088 15340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: No bootstrap required, opened a new log
Aug 09 03:35:40 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:40.771147 15340 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42451 } }
I20260809 03:35:40.771437 15340 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:40.771522 15340 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce3dd89e74c540c791b8d7dd7972ad84, State: Initialized, Role: FOLLOWER
I20260809 03:35:40.772104 15340 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42451 } }
I20260809 03:35:40.772320 15340 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:40.772423 15340 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:40.772584 15340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:40.773640 15340 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42451 } }
I20260809 03:35:40.774049 15340 leader_election.cc:302] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce3dd89e74c540c791b8d7dd7972ad84; no voters:
I20260809 03:35:40.774433 15340 leader_election.cc:288] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:40.774545 15349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:40.774845 15349 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 1 LEADER]: Becoming Leader. State: Replica: ce3dd89e74c540c791b8d7dd7972ad84, State: Running, Role: LEADER
I20260809 03:35:40.775195 15349 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42451 } }
I20260809 03:35:40.776026 15340 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:40.776628 15349 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce3dd89e74c540c791b8d7dd7972ad84. Latest consensus state: current_term: 1 leader_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42451 } } }
I20260809 03:35:40.776746 15349 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:40.777321 15354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:40.777549 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:40.761714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34990 (local address 127.1.224.126:42451)
0809 03:35:40.762186 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:40.762207 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:40.762251 (+ 44us) server_negotiation.cc:407] Connection header received
0809 03:35:40.763243 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:40.763271 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:40.763726 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:40.764113 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:40.765141 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:40.766427 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:40.767643 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:40.768141 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:40.768279 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:40.771474 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:40.771482 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:40.771504 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:40.771546 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:40.773560 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:40.774082 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:40.774093 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:40.774100 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:40.774280 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:40.774661 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:40.774664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:40.774666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:40.775274 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:40.775465 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:40.775973 (+ 508us) server_negotiation.cc:299] Negotiation successful
0809 03:35:40.776345 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":133,"threads_started":1}
I20260809 03:35:40.779120 15350 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42451 } } }
I20260809 03:35:40.779224 15350 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:40.781649 15354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:40.788201 15354 catalog_manager.cc:1347] Generated new cluster ID: 274c56e1d708494491a37afb513ab07a
I20260809 03:35:40.788295 15354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:40.812126 15354 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:40.813643 15354 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:40.820087 15354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: Generated new TSK 0
I20260809 03:35:40.820930 15354 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:35:40.857761 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:42451
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:40.980091 15375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:40.980387 15375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:40.980432 15375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:40.984551 15375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:40.984632 15375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:40.984691 15375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:40.990410 15375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:42451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:40.991706 15375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:40.992878 15375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:41.003387 15385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:41.003387 15382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:41.003520 15383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:41.003620 15375 server_base.cc:1034] running on GCE node
I20260809 03:35:41.004189 15375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:41.004963 15375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:41.006134 15375 hybrid_clock.cc:648] HybridClock initialized: now 1786246541006120 us; error 26 us; skew 500 ppm
Aug 09 03:35:41 dist-test-slave-jq4t krb5kdc[15213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:41.009194 15375 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:35:41.010175 15375 webserver.cc:460] Webserver started at http://127.1.224.65:42863/ using document root <none> and password file <none>
I20260809 03:35:41.010809 15375 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:41.010928 15375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:41.011188 15375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:41.013535 15375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "1959510d27684dd68fb5fd45e31ba9c1"
format_stamp: "Formatted at 2026-08-09 03:35:41 on dist-test-slave-jq4t"
server_key: "787cf6e2a2de5023992e6be657172f68"
server_key_iv: "a0bb8ccc679cc3cb0be1c335e19f15d2"
server_key_version: "encryptionkey@0"
I20260809 03:35:41.014176 15375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "1959510d27684dd68fb5fd45e31ba9c1"
format_stamp: "Formatted at 2026-08-09 03:35:41 on dist-test-slave-jq4t"
server_key: "787cf6e2a2de5023992e6be657172f68"
server_key_iv: "a0bb8ccc679cc3cb0be1c335e19f15d2"
server_key_version: "encryptionkey@0"
I20260809 03:35:41.018471 15375 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260809 03:35:41.021333 15394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:41.022634 15375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:35:41.022791 15375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "1959510d27684dd68fb5fd45e31ba9c1"
format_stamp: "Formatted at 2026-08-09 03:35:41 on dist-test-slave-jq4t"
server_key: "787cf6e2a2de5023992e6be657172f68"
server_key_iv: "a0bb8ccc679cc3cb0be1c335e19f15d2"
server_key_version: "encryptionkey@0"
I20260809 03:35:41.022924 15375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:41.043023 15375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:41.046743 15375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:41.046973 15375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:41.047636 15375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:41.048583 15375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:41.048668 15375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:41.048767 15375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:41.048820 15375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:41.058825 15375 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:46703
I20260809 03:35:41.058847 15559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:46703 every 8 connection(s)
I20260809 03:35:41.059877 15375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:41.067085 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 15375
I20260809 03:35:41.067209 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:41.067528 1921 external_mini_cluster.cc:1428] Setting key 5256dcc888f47a09b30441cc7d3d0542
Aug 09 03:35:41 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:41.076344 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:41.062200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:37329 (local address 127.1.224.126:42451)
0809 03:35:41.062354 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:41.062359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:41.063256 (+ 897us) server_negotiation.cc:407] Connection header received
0809 03:35:41.064713 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:41.064717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:41.064783 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:41.064868 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:41.066161 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:41.066731 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:41.068219 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:41.068602 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:41.068701 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:41.071821 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:41.071827 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:41.071832 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:41.071878 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:41.073678 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:41.074229 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:41.074236 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:41.074241 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:41.074343 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:41.074704 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:41.074711 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:41.074716 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:41.075041 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:41.075159 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:41.075832 (+ 673us) server_negotiation.cc:299] Negotiation successful
0809 03:35:41.076035 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:35:41.077239 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:41.062583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:37329)
0809 03:35:41.063089 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:41.063137 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:41.064416 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:41.065061 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:41.065069 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:41.065458 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:41.065986 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:41.066000 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:41.066883 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:41.066892 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:41.068061 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:41.068074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:41.068786 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:41.068790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:41.068999 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:41.069796 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:41.069813 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:41.071624 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:41.073821 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:41.073827 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:41.073843 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:41.074116 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:41.074442 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:41.074444 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:41.074445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:41.074598 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:41.075250 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:41.075256 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:41.075538 (+ 282us) client_negotiation.cc:769] Sending connection context
0809 03:35:41.075716 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 03:35:41.076089 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":134,"threads_started":1}
I20260809 03:35:41.078717 15560 heartbeater.cc:347] Connected to a master server at 127.1.224.126:42451
I20260809 03:35:41.078997 15560 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:41.079562 15560 heartbeater.cc:511] Master 127.1.224.126:42451 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:35:41.081369 15287 ts_manager.cc:194] Registered new tserver with Master: 1959510d27684dd68fb5fd45e31ba9c1 (127.1.224.65:46703)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:35:41.082826 15287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:37329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:35:41.099565 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:42451
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:41.220135 15565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:41.220580 15565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:41.220659 15565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:41.224789 15565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:41.224848 15565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:41.224947 15565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:41.230816 15565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:42451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:41.232030 15565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:41.233130 15565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:41.243247 15573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:41.243268 15575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:41.243244 15571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:41.243613 15565 server_base.cc:1034] running on GCE node
I20260809 03:35:41.244076 15565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:41.244814 15565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:41.246008 15565 hybrid_clock.cc:648] HybridClock initialized: now 1786246541245992 us; error 26 us; skew 500 ppm
Aug 09 03:35:41 dist-test-slave-jq4t krb5kdc[15213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:41.248780 15565 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:35:41.249720 15565 webserver.cc:460] Webserver started at http://127.1.224.66:45013/ using document root <none> and password file <none>
I20260809 03:35:41.250320 15565 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:41.250427 15565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:41.250680 15565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:41.252879 15565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "a02ee23c3cc44525ac07e0f538357ce6"
format_stamp: "Formatted at 2026-08-09 03:35:41 on dist-test-slave-jq4t"
server_key: "60068b2279be0344b2dea5aa43f4c334"
server_key_iv: "25288592d9e749f16dcc14cc3208ba7a"
server_key_version: "encryptionkey@0"
I20260809 03:35:41.253473 15565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "a02ee23c3cc44525ac07e0f538357ce6"
format_stamp: "Formatted at 2026-08-09 03:35:41 on dist-test-slave-jq4t"
server_key: "60068b2279be0344b2dea5aa43f4c334"
server_key_iv: "25288592d9e749f16dcc14cc3208ba7a"
server_key_version: "encryptionkey@0"
I20260809 03:35:41.257682 15565 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:35:41.260169 15584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:41.261467 15565 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:41.261641 15565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "a02ee23c3cc44525ac07e0f538357ce6"
format_stamp: "Formatted at 2026-08-09 03:35:41 on dist-test-slave-jq4t"
server_key: "60068b2279be0344b2dea5aa43f4c334"
server_key_iv: "25288592d9e749f16dcc14cc3208ba7a"
server_key_version: "encryptionkey@0"
I20260809 03:35:41.261768 15565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:41.277843 15565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:41.281101 15565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:41.281322 15565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:41.281963 15565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:41.282850 15565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:41.282928 15565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:41.283005 15565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:41.283058 15565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:41.292366 15565 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:34093
I20260809 03:35:41.292392 15749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:34093 every 8 connection(s)
I20260809 03:35:41.293398 15565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:41.298030 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 15565
I20260809 03:35:41.298170 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:41.298496 1921 external_mini_cluster.cc:1428] Setting key 4a2ca1085394296e98f48f8069dee91e
Aug 09 03:35:41 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:41.309511 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:41.295377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:56369 (local address 127.1.224.126:42451)
0809 03:35:41.295562 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:41.295567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:41.296410 (+ 843us) server_negotiation.cc:407] Connection header received
0809 03:35:41.297782 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:41.297784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:41.297836 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:41.297911 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:41.299542 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:41.300225 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:41.302144 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:41.302509 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:41.302638 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:41.305634 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:41.305638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:41.305640 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:41.305666 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:41.307198 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:41.307756 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:41.307760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:41.307761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:41.307838 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:41.308198 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:41.308201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:41.308203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:41.308425 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:41.308569 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:41.309163 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 03:35:41.309315 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:35:41.310460 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:41.295747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:56369)
0809 03:35:41.296247 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:41.296293 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:41.297547 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:41.298156 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:41.298166 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:41.298683 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:41.299349 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:41.299370 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:41.300368 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:41.300372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:41.301976 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:41.301993 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:41.302692 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:41.302697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:41.302896 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:41.303592 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:41.303610 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:41.305439 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:41.307333 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:41.307340 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:41.307358 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:41.307633 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:41.307933 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:41.307937 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:41.307939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:41.308090 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:41.308565 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:41.308571 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:41.308894 (+ 323us) client_negotiation.cc:769] Sending connection context
0809 03:35:41.309105 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 03:35:41.309484 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":134,"threads_started":1}
I20260809 03:35:41.311870 15750 heartbeater.cc:347] Connected to a master server at 127.1.224.126:42451
I20260809 03:35:41.312204 15750 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
I20260809 03:35:41.312891 15750 heartbeater.cc:511] Master 127.1.224.126:42451 requested a full tablet report, sending...
I20260809 03:35:41.314221 15287 ts_manager.cc:194] Registered new tserver with Master: a02ee23c3cc44525ac07e0f538357ce6 (127.1.224.66:34093)
I20260809 03:35:41.315132 15287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:56369
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:35:41.331629 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:42451
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:41.453385 15758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:41.453696 15758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:41.453775 15758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:41.457809 15758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:41.457901 15758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:41.457998 15758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:41.463263 15758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:42451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:41.464736 15758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:41.466002 15758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:41.477258 15767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:41.477233 15765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:41.477233 15770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:41.477528 15758 server_base.cc:1034] running on GCE node
I20260809 03:35:41.478053 15758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:41.478749 15758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:41.479921 15758 hybrid_clock.cc:648] HybridClock initialized: now 1786246541479907 us; error 27 us; skew 500 ppm
Aug 09 03:35:41 dist-test-slave-jq4t krb5kdc[15213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:41.482738 15758 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:35:41.483697 15758 webserver.cc:460] Webserver started at http://127.1.224.67:41635/ using document root <none> and password file <none>
I20260809 03:35:41.484292 15758 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:41.484403 15758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:41.484715 15758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:41.486825 15758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "767f03f0be844f0085ef97c0f114da67"
format_stamp: "Formatted at 2026-08-09 03:35:41 on dist-test-slave-jq4t"
server_key: "02f94afaee4043119a37054d3c26dcb1"
server_key_iv: "18a6b160f05fceaa7bfba87ef8310320"
server_key_version: "encryptionkey@0"
I20260809 03:35:41.487424 15758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "767f03f0be844f0085ef97c0f114da67"
format_stamp: "Formatted at 2026-08-09 03:35:41 on dist-test-slave-jq4t"
server_key: "02f94afaee4043119a37054d3c26dcb1"
server_key_iv: "18a6b160f05fceaa7bfba87ef8310320"
server_key_version: "encryptionkey@0"
I20260809 03:35:41.491477 15758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:35:41.494026 15784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:41.495388 15758 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:35:41.495543 15758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "767f03f0be844f0085ef97c0f114da67"
format_stamp: "Formatted at 2026-08-09 03:35:41 on dist-test-slave-jq4t"
server_key: "02f94afaee4043119a37054d3c26dcb1"
server_key_iv: "18a6b160f05fceaa7bfba87ef8310320"
server_key_version: "encryptionkey@0"
I20260809 03:35:41.495664 15758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:41.510485 15758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:41.513661 15758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:41.513885 15758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:41.514524 15758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:41.515411 15758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:41.515486 15758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:41.515588 15758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:41.515645 15758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:41.525039 15758 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:43675
I20260809 03:35:41.525070 15961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:43675 every 8 connection(s)
I20260809 03:35:41.526084 15758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:41.531430 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 15758
I20260809 03:35:41.531600 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:41.531948 1921 external_mini_cluster.cc:1428] Setting key 28d360d0c46a693bb01d2f67160cf69b
Aug 09 03:35:41 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:35:41.540695 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:41.528331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:47253 (local address 127.1.224.126:42451)
0809 03:35:41.528636 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:41.528640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:41.529397 (+ 757us) server_negotiation.cc:407] Connection header received
0809 03:35:41.530814 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:41.530818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:41.530900 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:41.530980 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:41.532400 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:41.533246 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:41.534591 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:41.534812 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:41.534896 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:41.537492 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:41.537495 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:41.537497 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:41.537530 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:41.538800 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:41.539285 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:41.539287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:41.539288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:41.539365 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:41.539663 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:41.539665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:41.539666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:41.539851 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:41.539931 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:41.540419 (+ 488us) server_negotiation.cc:299] Negotiation successful
0809 03:35:41.540559 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203}
I20260809 03:35:41.541620 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:41.528759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:47253)
0809 03:35:41.529239 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:41.529284 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:41.530547 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:41.531181 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:41.531188 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:41.531582 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:41.532213 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:41.532232 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:41.533412 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:41.533415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:41.534473 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:41.534483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:41.534944 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:41.534947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:41.535102 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:41.535702 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:41.535719 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:41.537328 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:41.538905 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:41.538911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:41.538925 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:41.539194 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:41.539444 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:35:41.539446 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:35:41.539447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:35:41.539582 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:35:41.539939 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:41.539945 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:41.540205 (+ 260us) client_negotiation.cc:769] Sending connection context
0809 03:35:41.540367 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 03:35:41.540723 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":132,"threads_started":1}
I20260809 03:35:41.542856 15964 heartbeater.cc:347] Connected to a master server at 127.1.224.126:42451
I20260809 03:35:41.543107 15964 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:41.543588 15964 heartbeater.cc:511] Master 127.1.224.126:42451 requested a full tablet report, sending...
I20260809 03:35:41.544829 15287 ts_manager.cc:194] Registered new tserver with Master: 767f03f0be844f0085ef97c0f114da67 (127.1.224.67:43675)
I20260809 03:35:41.545509 15287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:47253
I20260809 03:35:41.546332 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab.
Entry for principal oryx/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab.
WARNING: no policy specified for oryx/127.1.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab.
Entry for principal oryx/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab.
WARNING: no policy specified for oryx/127.1.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab.
Entry for principal oryx/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab.
I20260809 03:35:41.630805 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 15375
I20260809 03:35:41.644663 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 15565
I20260809 03:35:41.659003 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 15758
I20260809 03:35:41.672847 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 15230
I20260809 03:35:41.686528 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:42451
--webserver_interface=127.1.224.126
--webserver_port=42669
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:42451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:35:41.807915 15978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:41.808256 15978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:41.808300 15978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:41.812460 15978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:41.812634 15978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:41.812657 15978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:41.812675 15978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:41.812692 15978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:41.818646 15978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:42451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:42451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:41.819948 15978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:41.821102 15978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:41.830641 15995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:41.830740 15978 server_base.cc:1034] running on GCE node
W20260809 03:35:41.830631 15992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:41.830631 15993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:41.831660 15978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:41.832881 15978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:41.834059 15978 hybrid_clock.cc:648] HybridClock initialized: now 1786246541834024 us; error 45 us; skew 500 ppm
Aug 09 03:35:41 dist-test-slave-jq4t krb5kdc[15213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:41.837028 15978 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:35:41.837954 15978 webserver.cc:460] Webserver started at http://127.1.224.126:42669/ using document root <none> and password file <none>
I20260809 03:35:41.838511 15978 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:41.838595 15978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:41.842586 15978 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:35:41.844863 16004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:41.846159 15978 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:41.846293 15978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "ce3dd89e74c540c791b8d7dd7972ad84"
format_stamp: "Formatted at 2026-08-09 03:35:40 on dist-test-slave-jq4t"
server_key: "266e170d3076ca43a597d077ada002d1"
server_key_iv: "73295f2eb1ad9207e4587b570d51d721"
server_key_version: "encryptionkey@0"
I20260809 03:35:41.846844 15978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:41.865597 15978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:41.868775 15978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:41.868988 15978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:41.876730 15978 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:42451
I20260809 03:35:41.876754 16076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:42451 every 8 connection(s)
I20260809 03:35:41.877933 15978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:41.882948 16077 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:35:41.884542 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 15978
I20260809 03:35:41.885039 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:46703
--local_ip_for_outbound_sockets=127.1.224.65
--tserver_master_addrs=127.1.224.126:42451
--webserver_port=42863
--webserver_interface=127.1.224.65
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.224.65
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:35:41.885063 16077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: Bootstrap starting.
I20260809 03:35:41.909763 16077 log.cc:824] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:41.917835 16077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:35:41.918298 16077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: Bootstrap complete.
I20260809 03:35:41.922953 16077 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42451 } }
I20260809 03:35:41.923395 16077 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce3dd89e74c540c791b8d7dd7972ad84, State: Initialized, Role: FOLLOWER
I20260809 03:35:41.923996 16077 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42451 } }
I20260809 03:35:41.924132 16077 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:41.924230 16077 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:41.924340 16077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:35:41.926474 16077 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42451 } }
I20260809 03:35:41.926858 16077 leader_election.cc:302] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce3dd89e74c540c791b8d7dd7972ad84; no voters:
I20260809 03:35:41.927273 16077 leader_election.cc:288] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 03:35:41.927377 16085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:35:41.927603 16085 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [term 2 LEADER]: Becoming Leader. State: Replica: ce3dd89e74c540c791b8d7dd7972ad84, State: Running, Role: LEADER
I20260809 03:35:41.927917 16085 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42451 } }
I20260809 03:35:41.928687 16077 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:41.929394 16088 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42451 } } }
I20260809 03:35:41.929543 16088 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:41.934536 16109 catalog_manager.cc:1259] Loaded cluster ID: 274c56e1d708494491a37afb513ab07a
I20260809 03:35:41.934612 16109 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: loading cluster ID for follower catalog manager: success
I20260809 03:35:41.934854 16092 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce3dd89e74c540c791b8d7dd7972ad84. Latest consensus state: current_term: 2 leader_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce3dd89e74c540c791b8d7dd7972ad84" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42451 } } }
I20260809 03:35:41.935062 16092 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:41.937191 16109 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: acquiring CA information for follower catalog manager: success
I20260809 03:35:41.938755 16109 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:35:41.938901 16098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:41.940618 16098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:41.941092 16098 catalog_manager.cc:1259] Loaded cluster ID: 274c56e1d708494491a37afb513ab07a
I20260809 03:35:41.941148 16098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:41.941857 16098 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:41.942363 16098 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ce3dd89e74c540c791b8d7dd7972ad84: Loaded TSK: 0
I20260809 03:35:41.943161 16098 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:35:42.019100 16081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:42.019367 16081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:42.019413 16081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:42.023846 16081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:42.023937 16081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:42.024071 16081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:42.030278 16081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.65.keytab
--principal=oryx/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:46703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=42863
--tserver_master_addrs=127.1.224.126:42451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:42.031437 16081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:42.032581 16081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:42.043895 16120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:42.043882 16117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:42.043882 16122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:42.044663 16081 server_base.cc:1034] running on GCE node
I20260809 03:35:42.045209 16081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:42.046139 16081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:42.047369 16081 hybrid_clock.cc:648] HybridClock initialized: now 1786246542047332 us; error 50 us; skew 500 ppm
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246542, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:42.051383 16081 init.cc:377] Logged in from keytab as oryx/127.1.224.65@KRBTEST.COM (short username oryx)
I20260809 03:35:42.052452 16081 webserver.cc:460] Webserver started at http://127.1.224.65:42863/ using document root <none> and password file <none>
I20260809 03:35:42.053236 16081 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:42.053319 16081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:42.057812 16081 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:35:42.060170 16131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:42.061561 16081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:42.061714 16081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "1959510d27684dd68fb5fd45e31ba9c1"
format_stamp: "Formatted at 2026-08-09 03:35:41 on dist-test-slave-jq4t"
server_key: "787cf6e2a2de5023992e6be657172f68"
server_key_iv: "a0bb8ccc679cc3cb0be1c335e19f15d2"
server_key_version: "encryptionkey@0"
I20260809 03:35:42.062318 16081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:42.085815 16081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:42.089056 16081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:42.089283 16081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:42.089957 16081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:42.090865 16081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:42.090942 16081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:42.091017 16081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:42.091060 16081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:42.100270 16081 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:46703
I20260809 03:35:42.100306 16296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:46703 every 8 connection(s)
I20260809 03:35:42.101464 16081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:42.104095 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 16081
I20260809 03:35:42.104624 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:34093
--local_ip_for_outbound_sockets=127.1.224.66
--tserver_master_addrs=127.1.224.126:42451
--webserver_port=45013
--webserver_interface=127.1.224.66
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.224.66
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:42.116345 16300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:42.103635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:38155)
0809 03:35:42.104136 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:42.104212 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:42.105668 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:42.107014 (+ 1346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:42.107021 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:42.107378 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:42.107845 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.107860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.109999 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.110002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.110956 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.110967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.112136 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.112139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.112283 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.113037 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:42.113055 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:42.115546 (+ 2491us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":151,"threads_started":1}
W20260809 03:35:42.116919 16299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:42.103484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:38155 (local address 127.1.224.126:42451)
0809 03:35:42.103961 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:42.103983 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:42.104347 (+ 364us) server_negotiation.cc:407] Connection header received
0809 03:35:42.106139 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:42.106158 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:42.106564 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:42.106933 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:42.108043 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.109754 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.111099 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.111953 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.112178 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.115884 (+ 3706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:38155: BlockingRecv error: recv got EOF from 127.1.224.65:38155 (error 108)
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":130,"threads_started":1}
W20260809 03:35:42.116919 16297 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:42451 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.224.126:42451: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:42.122939 16300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:42.117441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:38873)
0809 03:35:42.117699 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:42.117713 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:42.117831 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:42.118107 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:42.118110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:42.118287 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:42.118382 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.118387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.119142 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.119147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.119796 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.119805 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.120267 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.120270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.120368 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.120918 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:42.120935 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:42.122779 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179}
W20260809 03:35:42.123049 16299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:42.117547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:38873 (local address 127.1.224.126:42451)
0809 03:35:42.117672 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:42.117676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:42.117785 (+ 109us) server_negotiation.cc:407] Connection header received
0809 03:35:42.117939 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:42.117946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:42.117993 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:42.118089 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:42.118501 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.119028 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.119930 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.120152 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.120263 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.122918 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:38873: BlockingRecv error: recv got EOF from 127.1.224.65:38873 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:42.129516 16300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:42.123759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:40827)
0809 03:35:42.123924 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:42.123938 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:42.124007 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:42.124239 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:42.124242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:42.124415 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:42.124546 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.124554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.125275 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.125278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.126172 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.126184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.126611 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.126615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.126701 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.127200 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:42.127215 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:42.129256 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260809 03:35:42.129606 16299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:42.123841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:40827 (local address 127.1.224.126:42451)
0809 03:35:42.123963 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:42.123966 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:42.123979 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:42.124090 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:42.124091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:42.124128 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:42.124194 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:42.124669 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.125170 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.126294 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.126499 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.126578 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.129458 (+ 2880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:40827: BlockingRecv error: recv got EOF from 127.1.224.65:40827 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 03:35:42.129813 16297 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:35:42.245062 16301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:42.245501 16301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:42.245597 16301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:42.254328 16301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:42.254436 16301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:42.254571 16301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:42.261592 16301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.66.keytab
--principal=oryx/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:34093
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=45013
--tserver_master_addrs=127.1.224.126:42451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:42.262730 16301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:42.263814 16301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:42.273979 16312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:42.274056 16301 server_base.cc:1034] running on GCE node
W20260809 03:35:42.274062 16311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:42.273989 16314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:42.274739 16301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:42.275396 16301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:42.276572 16301 hybrid_clock.cc:648] HybridClock initialized: now 1786246542276552 us; error 33 us; skew 500 ppm
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246542, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:42.279631 16301 init.cc:377] Logged in from keytab as oryx/127.1.224.66@KRBTEST.COM (short username oryx)
I20260809 03:35:42.280704 16301 webserver.cc:460] Webserver started at http://127.1.224.66:45013/ using document root <none> and password file <none>
I20260809 03:35:42.281275 16301 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:42.281349 16301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:42.285462 16301 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:35:42.287597 16327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:42.288955 16301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:42.289073 16301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "a02ee23c3cc44525ac07e0f538357ce6"
format_stamp: "Formatted at 2026-08-09 03:35:41 on dist-test-slave-jq4t"
server_key: "60068b2279be0344b2dea5aa43f4c334"
server_key_iv: "25288592d9e749f16dcc14cc3208ba7a"
server_key_version: "encryptionkey@0"
I20260809 03:35:42.289641 16301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:42.330029 16301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:42.333357 16301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:42.333549 16301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:42.334194 16301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:42.335137 16301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:42.335187 16301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:42.335229 16301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:42.335250 16301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:42.344589 16301 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:34093
I20260809 03:35:42.344630 16496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:34093 every 8 connection(s)
I20260809 03:35:42.345721 16301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:42.354835 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 16301
I20260809 03:35:42.355322 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:43675
--local_ip_for_outbound_sockets=127.1.224.67
--tserver_master_addrs=127.1.224.126:42451
--webserver_port=41635
--webserver_interface=127.1.224.67
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.224.67
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:35:42.358135 16299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:42.347648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:53907 (local address 127.1.224.126:42451)
0809 03:35:42.347790 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:42.347794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:42.348642 (+ 848us) server_negotiation.cc:407] Connection header received
0809 03:35:42.350139 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:42.350143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:42.350213 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:42.350294 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:42.351769 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.352407 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.353662 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.353907 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.354029 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.357958 (+ 3929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:53907: BlockingRecv error: recv got EOF from 127.1.224.66:53907 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:35:42.358624 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:42.347964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:53907)
0809 03:35:42.348423 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:42.348467 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:42.349916 (+ 1449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:42.350484 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:42.350491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:42.351065 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:42.351585 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.351600 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.352607 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.352610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.353503 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.353513 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.354083 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.354087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.354269 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.354950 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:42.354976 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:42.357740 (+ 2764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":113,"threads_started":1}
W20260809 03:35:42.359261 16497 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:42451 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.224.126:42451: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:42.365185 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:42.359733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:53591)
0809 03:35:42.359953 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:42.359975 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:42.360051 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:42.360314 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:42.360317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:42.360449 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:42.360608 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.360613 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.361399 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.361404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.361956 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.361964 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.362436 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.362439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.362536 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.363071 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:42.363083 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:42.365027 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148}
W20260809 03:35:42.365345 16299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:42.359879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:53591 (local address 127.1.224.126:42451)
0809 03:35:42.360046 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:42.360053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:42.360072 (+ 19us) server_negotiation.cc:407] Connection header received
0809 03:35:42.360124 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:42.360130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:42.360190 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:42.360296 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:42.360724 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.361250 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.362083 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.362318 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.362435 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.365159 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:53591: BlockingRecv error: recv got EOF from 127.1.224.66:53591 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:42.371493 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:42.365898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:47957)
0809 03:35:42.366055 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:42.366072 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:42.366161 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:42.366447 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:42.366451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:42.366630 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:42.366768 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.366779 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.367551 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.367556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.368557 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.368568 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.368996 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.368999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.369092 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.369542 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:42.369557 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:42.371362 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 03:35:42.371755 16299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:42.366121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:47957 (local address 127.1.224.126:42451)
0809 03:35:42.366249 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:42.366252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:42.366263 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:35:42.366292 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:42.366294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:42.366328 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:42.366418 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:42.366895 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.367431 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.368667 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.368887 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.368966 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.371508 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:47957: BlockingRecv error: recv got EOF from 127.1.224.66:47957 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 03:35:42.371799 16497 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:35:42.481056 16502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:42.481369 16502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:42.481415 16502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:42.485756 16502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:42.485837 16502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:42.485899 16502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:42.491565 16502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/krb5kdc/oryx_127.1.224.67.keytab
--principal=oryx/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:43675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=41635
--tserver_master_addrs=127.1.224.126:42451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:42.492902 16502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:42.494108 16502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:42.504292 16517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:42.504292 16513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:42.504333 16515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:42.504720 16502 server_base.cc:1034] running on GCE node
I20260809 03:35:42.505225 16502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:42.505944 16502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:42.507153 16502 hybrid_clock.cc:648] HybridClock initialized: now 1786246542507125 us; error 40 us; skew 500 ppm
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246542, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:35:42.510143 16502 init.cc:377] Logged in from keytab as oryx/127.1.224.67@KRBTEST.COM (short username oryx)
I20260809 03:35:42.511205 16502 webserver.cc:460] Webserver started at http://127.1.224.67:41635/ using document root <none> and password file <none>
I20260809 03:35:42.511838 16502 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:42.511958 16502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:42.516266 16502 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:42.518579 16527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:42.519837 16502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:35:42.519994 16502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "767f03f0be844f0085ef97c0f114da67"
format_stamp: "Formatted at 2026-08-09 03:35:41 on dist-test-slave-jq4t"
server_key: "02f94afaee4043119a37054d3c26dcb1"
server_key_iv: "18a6b160f05fceaa7bfba87ef8310320"
server_key_version: "encryptionkey@0"
I20260809 03:35:42.520685 16502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:42.546285 16502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:42.549584 16502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:42.549821 16502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:42.550572 16502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:42.551548 16502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:42.551618 16502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:42.551689 16502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:42.551739 16502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:42.561599 16502 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:43675
I20260809 03:35:42.561618 16692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:43675 every 8 connection(s)
I20260809 03:35:42.562752 16502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:42.564452 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 16502
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:42.575517 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:42.565618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35002 (local address 127.1.224.126:42451)
0809 03:35:42.565891 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:42.565896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:42.565912 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:35:42.565963 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:42.565966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:42.566017 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:42.566123 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:42.567255 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.567819 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.569031 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.569260 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.569342 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.571159 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:42.571165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:42.571181 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:35:42.571212 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:42.573233 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:42.573729 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:42.573734 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:42.573739 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:42.573881 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:35:42.574170 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:35:42.574173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:35:42.574175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:35:42.574628 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:42.574863 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:42.575125 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 03:35:42.575360 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":105,"threads_started":1}
W20260809 03:35:42.576303 16299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:42.564945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:40081 (local address 127.1.224.126:42451)
0809 03:35:42.565128 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:42.565132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:42.566228 (+ 1096us) server_negotiation.cc:407] Connection header received
0809 03:35:42.568003 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:42.568006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:42.568048 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:42.568110 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:42.569737 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.570379 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.571849 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.572088 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.572203 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.576186 (+ 3983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:40081: BlockingRecv error: recv got EOF from 127.1.224.67:40081 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:35:42.577060 16698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:42.565505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:40081)
0809 03:35:42.566057 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:42.566133 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:42.567745 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:42.568315 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:42.568325 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:42.568857 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:42.569551 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.569575 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.570528 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.570533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.571707 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.571718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.572253 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.572257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.572443 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.573140 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:42.573163 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:42.576080 (+ 2917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":152,"threads_started":1}
W20260809 03:35:42.577610 16693 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:42451 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.224.126:42451: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:42.582803 16698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:42.578092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:35547)
0809 03:35:42.578229 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:42.578245 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:42.578323 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:42.578551 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:42.578553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:42.578686 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:42.578779 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.578784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.579464 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.579466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.580012 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.580019 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.580555 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.580558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.580640 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.581054 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:42.581064 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:42.582708 (+ 1644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 03:35:42.582983 16697 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:42.578199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:35547 (local address 127.1.224.126:42451)
0809 03:35:42.578316 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:42.578322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:42.578339 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:35:42.578401 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:42.578404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:42.578450 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:42.578534 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:42.578879 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.579373 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.580111 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.580392 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.580545 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.582849 (+ 2304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:35547: BlockingRecv error: recv got EOF from 127.1.224.67:35547 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:42 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:42.588076 16698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:42.583390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:34215)
0809 03:35:42.583503 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:42.583515 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:42.583609 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:42.583875 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:42.583878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:42.584032 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:42.584100 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.584105 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.584897 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.584902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.585411 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:42.585418 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.585873 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.585877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:42.585961 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.586349 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:42.586359 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:42.587926 (+ 1567us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 03:35:42.588146 16697 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:42.583490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:34215 (local address 127.1.224.126:42451)
0809 03:35:42.583684 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:42.583688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:42.583700 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:35:42.583731 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:42.583734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:42.583774 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:42.583858 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:42.584213 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.584790 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.585535 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:42.585758 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:42.585863 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:42.588037 (+ 2174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:34215: BlockingRecv error: recv got EOF from 127.1.224.67:34215 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
W20260809 03:35:42.588320 16693 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 03:35:43 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:43 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:43.136323 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:43.130528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:39877)
0809 03:35:43.130815 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:43.130840 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:43.130935 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:43.131195 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:43.131198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:43.131389 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:43.131515 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:43.131521 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:43.132310 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:43.132313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:43.133009 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:43.133020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:43.133449 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:43.133452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:43.133530 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:43.134067 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:43.134079 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:43.136191 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":93,"threads_started":1}
W20260809 03:35:43.136545 16701 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:43.130588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:39877 (local address 127.1.224.126:42451)
0809 03:35:43.130893 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:43.130897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:43.130910 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:43.131029 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:43.131031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:43.131076 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:43.131154 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:43.131622 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:43.132175 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:43.133124 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:43.133346 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:43.133422 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:43.136364 (+ 2942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:39877: BlockingRecv error: recv got EOF from 127.1.224.65:39877 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":106,"threads_started":1}
Aug 09 03:35:43 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:43 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:43.378814 16703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:43.372683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:59775)
0809 03:35:43.372978 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:43.372993 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:43.373096 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:43.373328 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:43.373330 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:43.373538 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:43.373649 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:43.373655 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:43.374502 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:43.374505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:43.375110 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:43.375118 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:43.375626 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:43.375630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:43.375701 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:43.376324 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:43.376337 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:43.378604 (+ 2267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":84,"threads_started":1}
W20260809 03:35:43.378955 16701 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:43.372757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:59775 (local address 127.1.224.126:42451)
0809 03:35:43.372917 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:43.372923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:43.373065 (+ 142us) server_negotiation.cc:407] Connection header received
0809 03:35:43.373190 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:43.373192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:43.373239 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:43.373310 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:43.373794 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:43.374360 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:43.375231 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:43.375492 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:43.375585 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:43.378831 (+ 3246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:59775: BlockingRecv error: recv got EOF from 127.1.224.66:59775 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 03:35:43 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:43 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:43.595119 16704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:43.589065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:54881)
0809 03:35:43.589365 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:43.589380 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:43.589491 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:43.589769 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:43.589772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:43.589974 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:43.590101 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:43.590107 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:43.590971 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:43.590973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:43.591651 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:43.591662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:43.592200 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:43.592204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:43.592309 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:43.592999 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:43.593014 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:43.594995 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":79,"threads_started":1}
W20260809 03:35:43.595297 16701 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:43.589165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:54881 (local address 127.1.224.126:42451)
0809 03:35:43.589366 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:43.589370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:43.589462 (+ 92us) server_negotiation.cc:407] Connection header received
0809 03:35:43.589580 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:43.589583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:43.589632 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:43.589704 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:43.590251 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:43.590815 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:43.591794 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:43.592072 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:43.592194 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:43.595139 (+ 2945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:54881: BlockingRecv error: recv got EOF from 127.1.224.67:54881 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 09 03:35:44 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:44 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:44.143100 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:44.137283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:39421)
0809 03:35:44.137625 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:44.137643 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:44.137796 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:44.138099 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:44.138102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:44.138331 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:44.138434 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:44.138440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.139254 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.139260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:44.139840 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:44.139848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.140353 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.140357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:44.140449 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:44.141013 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:44.141025 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:44.142902 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":115,"threads_started":1}
W20260809 03:35:44.143213 16708 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:44.137344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:39421 (local address 127.1.224.126:42451)
0809 03:35:44.137627 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:44.137633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:44.137731 (+ 98us) server_negotiation.cc:407] Connection header received
0809 03:35:44.137905 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:44.137909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:44.137968 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:44.138079 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:44.138561 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.139132 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.139966 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.140223 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.140337 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:44.143067 (+ 2730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:39421: BlockingRecv error: recv got EOF from 127.1.224.65:39421 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":69,"threads_started":1}
Aug 09 03:35:44 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:44 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:44.386238 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:44.380030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:40185)
0809 03:35:44.380350 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:44.380369 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:44.380600 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:44.380927 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:44.380930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:44.381153 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:44.381261 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:44.381266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.382081 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.382087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:44.382660 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:44.382669 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.383161 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.383165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:44.383260 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:44.383837 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:44.383849 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:44.386008 (+ 2159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":99,"threads_started":1}
W20260809 03:35:44.386368 16708 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:44.380209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:40185 (local address 127.1.224.126:42451)
0809 03:35:44.380353 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:44.380360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:44.380497 (+ 137us) server_negotiation.cc:407] Connection header received
0809 03:35:44.380725 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:44.380728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:44.380790 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:44.380890 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:44.381391 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.381959 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.382785 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.383031 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.383153 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:44.386223 (+ 3070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:40185: BlockingRecv error: recv got EOF from 127.1.224.66:40185 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 03:35:44 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:44 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:44.601352 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:44.595962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:54771)
0809 03:35:44.596241 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:44.596256 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:44.596347 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:44.596629 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:44.596631 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:44.596818 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:44.596918 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:44.596923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.597685 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.597688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:44.598261 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:44.598269 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.598691 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.598693 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:44.598758 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:44.599293 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:44.599305 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:44.601225 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":76,"threads_started":1}
W20260809 03:35:44.601531 16708 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:44.596119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:54771 (local address 127.1.224.126:42451)
0809 03:35:44.596308 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:44.596312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:44.596327 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:35:44.596441 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:44.596443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:44.596517 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:44.596590 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:44.597025 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.597575 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.598363 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:44.598589 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:44.598671 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:44.601391 (+ 2720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:54771: BlockingRecv error: recv got EOF from 127.1.224.67:54771 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 09 03:35:45 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:45 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:45.150336 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.144172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:60201)
0809 03:35:45.144532 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:45.144551 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:45.144665 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:45.144947 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:45.144949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:45.145154 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:45.145266 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.145271 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.146120 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.146123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.146714 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.146722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.147184 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.147188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.147257 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.147802 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:45.147814 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:45.150132 (+ 2318us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":108,"threads_started":1}
W20260809 03:35:45.150467 16715 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:45.144342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:60201 (local address 127.1.224.126:42451)
0809 03:35:45.144630 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.144634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.144649 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:35:45.144761 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.144763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.144810 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.144904 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:45.145421 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.146011 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.146823 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.147068 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.147147 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.150315 (+ 3168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:60201: BlockingRecv error: recv got EOF from 127.1.224.65:60201 (error 108)
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":143,"threads_started":1}
Aug 09 03:35:45 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:45 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:45.393988 16716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.387283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:40007)
0809 03:35:45.387573 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:45.387591 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:45.387715 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:45.388047 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:45.388051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:45.388306 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:45.388465 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.388504 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.389709 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.389712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.390300 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.390310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.390760 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.390763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.390840 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.391354 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:45.391366 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:45.393803 (+ 2437us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":103,"threads_started":1}
W20260809 03:35:45.394066 16715 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:45.387434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:40007 (local address 127.1.224.126:42451)
0809 03:35:45.387589 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.387593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.387659 (+ 66us) server_negotiation.cc:407] Connection header received
0809 03:35:45.387791 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.387793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.387860 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.387967 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:45.388661 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.389527 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.390419 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.390648 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.390733 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.393942 (+ 3209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:40007: BlockingRecv error: recv got EOF from 127.1.224.66:40007 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 03:35:45 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:45 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:45.608404 16717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:45.602235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:56137)
0809 03:35:45.602587 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:45.602607 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:45.602757 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:45.603052 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:45.603055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:45.603280 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:45.603381 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.603387 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.604189 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.604193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.604825 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:45.604834 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.605330 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.605333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:45.605419 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.605969 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:45.605980 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:45.608210 (+ 2230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":96,"threads_started":1}
W20260809 03:35:45.608548 16715 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:45.602374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:56137 (local address 127.1.224.126:42451)
0809 03:35:45.602564 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:45.602569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:45.602694 (+ 125us) server_negotiation.cc:407] Connection header received
0809 03:35:45.602863 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:45.602867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:45.602929 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:45.603025 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:45.603511 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.604071 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.604961 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:45.605209 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:45.605326 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:45.608389 (+ 3063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:56137: BlockingRecv error: recv got EOF from 127.1.224.67:56137 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 03:35:46 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:46 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:46.159919 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:46.151317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:56451)
0809 03:35:46.151615 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:46.151630 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:46.151733 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:46.151987 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:46.151989 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:46.152178 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:46.152284 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:46.152289 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:46.153048 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:46.153050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:46.153620 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:46.153628 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:46.154093 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:46.154095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:46.154161 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:46.157668 (+ 3507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:46.157682 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:46.159784 (+ 2102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":98,"threads_started":1}
W20260809 03:35:46.160199 16720 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:46.151327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:56451 (local address 127.1.224.126:42451)
0809 03:35:46.151642 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:46.151646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:46.151684 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:35:46.151820 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:46.151822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:46.151869 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:46.151949 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:46.152387 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:46.152955 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:46.153737 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:46.153976 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:46.154074 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:46.160040 (+ 5966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:56451: BlockingRecv error: recv got EOF from 127.1.224.65:56451 (error 108)
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":102,"threads_started":1}
Aug 09 03:35:46 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:46 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:46.400466 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:46.394801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:56853)
0809 03:35:46.395146 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:46.395160 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:46.395269 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:46.395520 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:46.395522 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:46.395712 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:46.395815 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:46.395821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:46.396640 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:46.396643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:46.397213 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:46.397223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:46.397658 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:46.397661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:46.397731 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:46.398268 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:46.398280 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:46.400303 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":89,"threads_started":1}
W20260809 03:35:46.400595 16720 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:46.394935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:56853 (local address 127.1.224.126:42451)
0809 03:35:46.395100 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:46.395104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:46.395233 (+ 129us) server_negotiation.cc:407] Connection header received
0809 03:35:46.395353 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:46.395355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:46.395404 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:46.395479 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:46.395923 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:46.396502 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:46.397323 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:46.397555 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:46.397636 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:46.400421 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:56853: BlockingRecv error: recv got EOF from 127.1.224.66:56853 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 03:35:46 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:46 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:46.615209 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:46.609323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:55855)
0809 03:35:46.609691 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:46.609707 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:46.609851 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:46.610151 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:46.610154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:46.610376 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:46.610475 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:46.610482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:46.611318 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:46.611323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:46.611901 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:46.611909 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:46.612411 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:46.612415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:46.612541 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:46.613148 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:46.613162 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:46.615065 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":121,"threads_started":1}
W20260809 03:35:46.615386 16720 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:46.609475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:55855 (local address 127.1.224.126:42451)
0809 03:35:46.609648 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:46.609653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:46.609790 (+ 137us) server_negotiation.cc:407] Connection header received
0809 03:35:46.609970 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:46.609973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:46.610036 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:46.610125 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:46.610609 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:46.611198 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:46.612042 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:46.612283 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:46.612396 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:46.615258 (+ 2862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:55855: BlockingRecv error: recv got EOF from 127.1.224.67:55855 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 03:35:47 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:47 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:47.167200 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:47.161135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:48465)
0809 03:35:47.161451 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:47.161478 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:47.161577 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:47.161886 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:47.161889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:47.162108 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:47.162213 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:47.162219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:47.163090 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:47.163093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:47.163695 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:47.163704 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:47.164237 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:47.164240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:47.164312 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:47.165007 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:47.165021 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:47.167070 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":106,"threads_started":1}
W20260809 03:35:47.167419 16724 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:47.161239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:48465 (local address 127.1.224.126:42451)
0809 03:35:47.161560 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:47.161564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:47.161581 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:35:47.161659 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:47.161661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:47.161706 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:47.161801 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:47.162357 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:47.162956 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:47.163829 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:47.164106 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:47.164204 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:47.167230 (+ 3026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:48465: BlockingRecv error: recv got EOF from 127.1.224.65:48465 (error 108)
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":134,"threads_started":1}
Aug 09 03:35:47 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:47 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:47.407542 16725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:47.401475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:43589)
0809 03:35:47.401829 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:47.401848 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:47.401979 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:47.402278 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:47.402281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:47.402519 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:47.402623 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:47.402630 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:47.403449 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:47.403454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:47.404030 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:47.404038 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:47.404545 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:47.404549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:47.404639 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:47.405187 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:47.405200 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:47.407370 (+ 2170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":113,"threads_started":1}
W20260809 03:35:47.407711 16724 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:47.401686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:43589 (local address 127.1.224.126:42451)
0809 03:35:47.401860 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:47.401896 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:47.401916 (+ 20us) server_negotiation.cc:407] Connection header received
0809 03:35:47.402102 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:47.402105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:47.402161 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:47.402253 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:47.402750 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:47.403329 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:47.404160 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:47.404403 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:47.404534 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:47.407556 (+ 3022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:43589: BlockingRecv error: recv got EOF from 127.1.224.66:43589 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 03:35:47 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:47 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:47.622653 16726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:47.616205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:51563)
0809 03:35:47.616567 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:47.616586 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:47.616727 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:47.617032 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:47.617035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:47.617267 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:47.617364 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:47.617370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:47.618184 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:47.618189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:47.618786 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:47.618794 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:47.619285 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:47.619289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:47.619377 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:47.619969 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:47.619982 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:47.622415 (+ 2433us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":98,"threads_started":1}
W20260809 03:35:47.622913 16724 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:47.616315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:51563 (local address 127.1.224.126:42451)
0809 03:35:47.616535 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:47.616540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:47.616673 (+ 133us) server_negotiation.cc:407] Connection header received
0809 03:35:47.616854 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:47.616857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:47.616906 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:47.616998 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:47.617501 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:47.618065 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:47.618912 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:47.619158 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:47.619278 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:47.622719 (+ 3441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:51563: BlockingRecv error: recv got EOF from 127.1.224.67:51563 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 03:35:48 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:48 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:48.174232 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:48.168187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:35719)
0809 03:35:48.168579 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:48.168596 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:48.168730 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:48.169054 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:48.169058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:48.169317 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:48.169453 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:48.169461 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:48.170268 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:48.170271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:48.170852 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:48.170860 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:48.171316 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:48.171319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:48.171386 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:48.171982 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:48.171995 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:48.174102 (+ 2107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":91,"threads_started":1}
W20260809 03:35:48.174479 16728 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:48.168256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:35719 (local address 127.1.224.126:42451)
0809 03:35:48.168590 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:48.168594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:48.168738 (+ 144us) server_negotiation.cc:407] Connection header received
0809 03:35:48.168825 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:48.168830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:48.168901 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:48.168989 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:48.169586 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:48.170154 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:48.170966 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:48.171203 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:48.171286 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:48.174310 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:35719: BlockingRecv error: recv got EOF from 127.1.224.65:35719 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":109,"threads_started":1}
Aug 09 03:35:48 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:48 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:48.415045 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:48.408851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:50031)
0809 03:35:48.409316 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:48.409330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:48.409490 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:48.409794 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:48.409797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:48.410051 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:48.410159 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:48.410165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:48.411033 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:48.411039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:48.411615 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:48.411624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:48.412139 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:48.412142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:48.412236 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:48.412828 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:48.412840 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:48.414903 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":105,"threads_started":1}
W20260809 03:35:48.415249 16728 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:48.408952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:50031 (local address 127.1.224.126:42451)
0809 03:35:48.409150 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:48.409154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:48.409429 (+ 275us) server_negotiation.cc:407] Connection header received
0809 03:35:48.409600 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:48.409604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:48.409664 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:48.409757 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:48.410293 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:48.410901 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:48.411751 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:48.412002 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:48.412130 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:48.415078 (+ 2948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:50031: BlockingRecv error: recv got EOF from 127.1.224.66:50031 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 09 03:35:48 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:48 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:48.630427 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:48.623937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:53439)
0809 03:35:48.624251 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:48.624267 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:48.624413 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:48.624791 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:48.624794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:48.625120 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:48.625259 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:48.625267 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:48.626392 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:48.626394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:48.627052 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:48.627062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:48.627547 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:48.627549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:48.627614 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:48.628180 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:48.628192 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:48.630300 (+ 2108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":109,"threads_started":1}
W20260809 03:35:48.630683 16728 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:48.624032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:53439 (local address 127.1.224.126:42451)
0809 03:35:48.624227 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:48.624233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:48.624353 (+ 120us) server_negotiation.cc:407] Connection header received
0809 03:35:48.624579 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:48.624581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:48.624637 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:48.624712 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:48.625405 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:48.626266 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:48.627194 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:48.627428 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:48.627519 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:48.630553 (+ 3034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:53439: BlockingRecv error: recv got EOF from 127.1.224.67:53439 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 09 03:35:49 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:49 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:49.181155 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:49.175067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:40055)
0809 03:35:49.175479 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:49.175494 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:49.175599 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:49.175878 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:49.175880 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:49.176073 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:49.176176 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:49.176181 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:49.177059 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:49.177063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:49.177663 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:49.177672 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:49.178148 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:49.178151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:49.178224 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:49.178863 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:49.178877 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:49.180965 (+ 2088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":92,"threads_started":1}
W20260809 03:35:49.181222 16736 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:49.175220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:40055 (local address 127.1.224.126:42451)
0809 03:35:49.175490 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:49.175494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:49.175549 (+ 55us) server_negotiation.cc:407] Connection header received
0809 03:35:49.175708 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:49.175711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:49.175756 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:49.175834 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:49.176287 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:49.176922 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:49.177772 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:49.178006 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:49.178084 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:49.181087 (+ 3003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:40055: BlockingRecv error: recv got EOF from 127.1.224.65:40055 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":65,"threads_started":1}
Aug 09 03:35:49 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:49 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:49.422179 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:49.415985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:52551)
0809 03:35:49.416442 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:49.416457 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:49.416644 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:49.416938 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:49.416941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:49.417172 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:49.417275 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:49.417281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:49.418091 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:49.418096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:49.418668 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:49.418677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:49.419257 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:49.419260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:49.419339 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:49.420064 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:49.420076 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:49.422051 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":121,"threads_started":1}
W20260809 03:35:49.422366 16736 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:49.416151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:52551 (local address 127.1.224.126:42451)
0809 03:35:49.416383 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:49.416389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:49.416581 (+ 192us) server_negotiation.cc:407] Connection header received
0809 03:35:49.416757 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:49.416760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:49.416821 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:49.416912 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:49.417404 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:49.417966 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:49.418807 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:49.419110 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:49.419220 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:49.422204 (+ 2984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:52551: BlockingRecv error: recv got EOF from 127.1.224.66:52551 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 09 03:35:49 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:49 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:49.637365 16739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:49.631441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:46973)
0809 03:35:49.631785 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:49.631798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:49.631902 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:49.632188 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:49.632190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:49.632383 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:49.632516 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:49.632522 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:49.633266 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:49.633269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:49.633822 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:49.633829 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:49.634347 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:49.634351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:49.634446 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:49.634961 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:49.634973 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:49.637186 (+ 2213us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":96,"threads_started":1}
W20260809 03:35:49.637441 16736 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:49.631502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:46973 (local address 127.1.224.126:42451)
0809 03:35:49.631658 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:49.631662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:49.631882 (+ 220us) server_negotiation.cc:407] Connection header received
0809 03:35:49.631993 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:49.631996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:49.632058 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:49.632152 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:49.632616 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:49.633169 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:49.633925 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:49.634220 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:49.634336 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:49.637315 (+ 2979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:46973: BlockingRecv error: recv got EOF from 127.1.224.67:46973 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 03:35:50 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:50 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:50.187662 16740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:50.182116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:56017)
0809 03:35:50.182471 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:50.182486 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:50.182589 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:50.182884 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:50.182887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:50.183080 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:50.183183 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:50.183188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:50.183934 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:50.183937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:50.184542 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:50.184550 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:50.184964 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:50.184966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:50.185029 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:50.185657 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:50.185670 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:50.187538 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":116,"threads_started":1}
W20260809 03:35:50.187881 16743 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:50.182343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:56017 (local address 127.1.224.126:42451)
0809 03:35:50.182663 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:50.182666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:50.182679 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:50.182716 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:50.182719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:50.182760 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:50.182836 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:50.183291 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:50.183841 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:50.184640 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:50.184864 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:50.184958 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:50.187694 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:56017: BlockingRecv error: recv got EOF from 127.1.224.65:56017 (error 108)
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":162,"threads_started":1}
Aug 09 03:35:50 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:50 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:50.429729 16746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:50.423097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:47533)
0809 03:35:50.423509 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:50.423551 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:50.423707 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:50.424045 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:50.424048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:50.424279 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:50.424382 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:50.424387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:50.425502 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:50.425508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:50.426121 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:50.426131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:50.426692 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:50.426696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:50.426799 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:50.427478 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:50.427493 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:50.429612 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":122,"threads_started":1}
W20260809 03:35:50.429934 16743 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:50.423258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:47533 (local address 127.1.224.126:42451)
0809 03:35:50.423500 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:50.423506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:50.423650 (+ 144us) server_negotiation.cc:407] Connection header received
0809 03:35:50.423835 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:50.423839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:50.423904 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:50.424007 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:50.424553 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:50.425389 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:50.426246 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:50.426537 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:50.426637 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:50.429740 (+ 3103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:47533: BlockingRecv error: recv got EOF from 127.1.224.66:47533 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 09 03:35:50 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:50 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:50.644120 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:50.638236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:44537)
0809 03:35:50.638584 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:50.638598 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:50.638736 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:50.639034 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:50.639037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:50.639288 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:50.639412 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:50.639427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:50.640213 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:50.640217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:50.640877 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:50.640893 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:50.641367 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:50.641370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:50.641459 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:50.642003 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:50.642016 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:50.643941 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":105,"threads_started":1}
W20260809 03:35:50.644189 16743 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:50.638344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:44537 (local address 127.1.224.126:42451)
0809 03:35:50.638499 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:50.638504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:50.638677 (+ 173us) server_negotiation.cc:407] Connection header received
0809 03:35:50.638863 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:50.638867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:50.638916 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:50.639007 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:50.639541 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:50.640096 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:50.641022 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:50.641243 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:50.641352 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:50.644075 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:44537: BlockingRecv error: recv got EOF from 127.1.224.67:44537 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 03:35:51 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:51 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:51.197540 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:51.188598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:33131)
0809 03:35:51.188926 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:51.188940 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:51.189048 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:51.189289 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:51.189292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:51.189493 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:51.189605 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:51.189610 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:51.190436 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:51.190439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:51.191050 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:51.191058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:51.191526 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:51.191528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:51.191592 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:51.194909 (+ 3317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:51.194926 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:51.197364 (+ 2438us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":109,"threads_started":1}
W20260809 03:35:51.197682 16752 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:51.188629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:33131 (local address 127.1.224.126:42451)
0809 03:35:51.188903 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:51.188907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:51.188997 (+ 90us) server_negotiation.cc:407] Connection header received
0809 03:35:51.189120 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:51.189122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:51.189167 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:51.189246 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:51.189737 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:51.190309 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:51.191172 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:51.191414 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:51.191511 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:51.197540 (+ 6029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:33131: BlockingRecv error: recv got EOF from 127.1.224.65:33131 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":90,"threads_started":1}
Aug 09 03:35:51 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:51 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:51.436561 16753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:51.430588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:43097)
0809 03:35:51.430888 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:51.430904 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:51.431011 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:51.431279 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:51.431281 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:51.431473 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:51.431575 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:51.431580 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:51.432352 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:51.432355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:51.432976 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:51.432985 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:51.433417 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:51.433420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:51.433485 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:51.434084 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:51.434096 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:51.436343 (+ 2247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":104,"threads_started":1}
W20260809 03:35:51.436635 16752 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:51.430742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:43097 (local address 127.1.224.126:42451)
0809 03:35:51.430927 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:51.430931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:51.430951 (+ 20us) server_negotiation.cc:407] Connection header received
0809 03:35:51.431105 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:51.431108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:51.431158 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:51.431242 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:51.431681 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:51.432249 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:51.433085 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:51.433311 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:51.433397 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:51.436535 (+ 3138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:43097: BlockingRecv error: recv got EOF from 127.1.224.66:43097 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 03:35:51 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:51 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:51.651183 16755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:51.645108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:50769)
0809 03:35:51.645409 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:51.645422 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:51.645538 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:51.645775 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:51.645777 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:51.645958 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:51.646059 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:51.646064 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:51.646773 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:51.646775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:51.647427 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:51.647437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:51.648092 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:51.648096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:51.648187 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:51.648907 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:51.648926 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:51.651004 (+ 2078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":78,"threads_started":1}
W20260809 03:35:51.651273 16752 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:51.645202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:50769 (local address 127.1.224.126:42451)
0809 03:35:51.645394 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:51.645399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:51.645495 (+ 96us) server_negotiation.cc:407] Connection header received
0809 03:35:51.645620 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:51.645623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:51.645670 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:51.645741 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:51.646154 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:51.646684 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:51.647607 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:51.647947 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:51.648089 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:51.651150 (+ 3061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:50769: BlockingRecv error: recv got EOF from 127.1.224.67:50769 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 09 03:35:52 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:52 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:52.204532 16756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:52.198534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:46089)
0809 03:35:52.198834 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:52.198848 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:52.198951 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:52.199211 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:52.199213 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:52.199412 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:52.199514 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:52.199519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:52.200281 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:52.200284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:52.200913 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:52.200921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:52.201519 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:52.201522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:52.201591 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:52.202138 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:52.202150 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:52.204355 (+ 2205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":99,"threads_started":1}
W20260809 03:35:52.204720 16758 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:52.198593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:46089 (local address 127.1.224.126:42451)
0809 03:35:52.198859 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:52.198863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:52.198931 (+ 68us) server_negotiation.cc:407] Connection header received
0809 03:35:52.199040 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:52.199043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:52.199091 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:52.199174 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:52.199618 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:52.200173 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:52.201015 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:52.201361 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:52.201476 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:52.204569 (+ 3093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:46089: BlockingRecv error: recv got EOF from 127.1.224.65:46089 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":70,"threads_started":1}
Aug 09 03:35:52 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:52 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:52.443336 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:52.437379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:39533)
0809 03:35:52.437737 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:52.437761 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:52.437903 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:52.438210 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:52.438213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:52.438448 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:52.438552 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:52.438558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:52.439349 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:52.439354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:52.439928 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:52.439937 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:52.440431 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:52.440435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:52.440574 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:52.441126 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:52.441139 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:52.443209 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":122,"threads_started":1}
W20260809 03:35:52.443526 16758 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:52.437524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:39533 (local address 127.1.224.126:42451)
0809 03:35:52.437703 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:52.437708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:52.437843 (+ 135us) server_negotiation.cc:407] Connection header received
0809 03:35:52.438025 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:52.438030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:52.438086 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:52.438179 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:52.438680 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:52.439235 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:52.440069 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:52.440305 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:52.440415 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:52.443383 (+ 2968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:39533: BlockingRecv error: recv got EOF from 127.1.224.66:39533 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 03:35:52 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:52 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:52.657599 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:52.652087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:36837)
0809 03:35:52.652389 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:52.652404 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:52.652568 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:52.652821 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:52.652824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:52.653015 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:52.653119 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:52.653124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:52.653883 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:52.653885 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:52.654441 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:52.654457 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:52.654864 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:52.654867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:52.654926 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:52.655460 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:52.655473 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:52.657394 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":111,"threads_started":1}
W20260809 03:35:52.657665 16758 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:52.652233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:36837 (local address 127.1.224.126:42451)
0809 03:35:52.652399 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:52.652403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:52.652520 (+ 117us) server_negotiation.cc:407] Connection header received
0809 03:35:52.652659 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:52.652661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:52.652707 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:52.652779 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:52.653232 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:52.653785 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:52.654550 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:52.654763 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:52.654840 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:52.657539 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:36837: BlockingRecv error: recv got EOF from 127.1.224.67:36837 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 03:35:53 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:53 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:53.211941 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:53.205679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:45137)
0809 03:35:53.206000 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:53.206015 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:53.206158 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:53.206504 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:53.206509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:53.206773 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:53.206919 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:53.206926 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:53.207875 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:53.207879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:53.208669 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:53.208680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:53.209182 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:53.209186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:53.209278 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:53.209831 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:53.209844 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:53.211787 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":103,"threads_started":1}
W20260809 03:35:53.212141 16762 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:53.205767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:45137 (local address 127.1.224.126:42451)
0809 03:35:53.206132 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:53.206137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:53.206158 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:35:53.206282 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:53.206285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:53.206354 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:53.206488 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:53.207085 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:53.207748 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:53.208797 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:53.209057 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:53.209175 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:53.211958 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:45137: BlockingRecv error: recv got EOF from 127.1.224.65:45137 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":126,"threads_started":1}
Aug 09 03:35:53 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:53 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:53.450071 16763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:53.444143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:49453)
0809 03:35:53.444525 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:53.444543 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:53.444704 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:53.445001 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:53.445004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:53.445248 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:53.445379 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:53.445386 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:53.446179 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:53.446183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:53.446753 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:53.446761 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:53.447252 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:53.447256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:53.447345 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:53.447898 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:53.447911 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:53.449887 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":97,"threads_started":1}
W20260809 03:35:53.450160 16762 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:53.444294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:49453 (local address 127.1.224.126:42451)
0809 03:35:53.444518 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:53.444524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:53.444636 (+ 112us) server_negotiation.cc:407] Connection header received
0809 03:35:53.444824 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:53.444834 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:53.444888 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:53.444978 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:53.445506 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:53.446061 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:53.446885 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:53.447124 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:53.447240 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:53.450038 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:49453: BlockingRecv error: recv got EOF from 127.1.224.66:49453 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 09 03:35:53 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:53 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:53.665174 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:53.658578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:55955)
0809 03:35:53.658884 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:53.658901 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:53.659045 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:53.659330 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:53.659332 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:53.659562 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:53.659665 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:53.659670 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:53.660842 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:53.660845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:53.661632 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:53.661642 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:53.662150 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:53.662153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:53.662228 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:53.662767 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:53.662779 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:53.664980 (+ 2201us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":107,"threads_started":1}
W20260809 03:35:53.665271 16762 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:53.658638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:55955 (local address 127.1.224.126:42451)
0809 03:35:53.658816 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:53.658821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:53.658985 (+ 164us) server_negotiation.cc:407] Connection header received
0809 03:35:53.659156 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:53.659159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:53.659219 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:53.659307 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:53.659786 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:53.660704 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:53.661753 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:53.661999 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:53.662096 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:53.665141 (+ 3045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:55955: BlockingRecv error: recv got EOF from 127.1.224.67:55955 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 03:35:54 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:54 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:54.218586 16766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:54.213011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:49829)
0809 03:35:54.213302 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:54.213318 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:54.213443 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:54.213700 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:54.213702 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:54.213899 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:54.214007 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:54.214012 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:54.214827 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:54.214829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:54.215394 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:54.215401 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:54.215943 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:54.215946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:54.216020 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:54.216598 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:54.216610 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:54.218449 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":97,"threads_started":1}
W20260809 03:35:54.218762 16768 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:54.213068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:49829 (local address 127.1.224.126:42451)
0809 03:35:54.213328 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:54.213333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:54.213384 (+ 51us) server_negotiation.cc:407] Connection header received
0809 03:35:54.213525 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:54.213527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:54.213576 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:54.213659 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:54.214122 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:54.214724 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:54.215524 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:54.215795 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:54.215914 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:54.218611 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:49829: BlockingRecv error: recv got EOF from 127.1.224.65:49829 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":86,"threads_started":1}
Aug 09 03:35:54 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:54 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:54.457283 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:54.451061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:42943)
0809 03:35:54.451487 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:54.451505 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:54.451636 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:54.452131 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:54.452134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:54.452412 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:54.452587 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:54.452597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:54.453411 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:54.453414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:54.454070 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:54.454080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:54.454568 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:54.454571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:54.454636 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:54.455229 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:54.455247 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:54.457158 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":92,"threads_started":1}
W20260809 03:35:54.457486 16768 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:54.451201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:42943 (local address 127.1.224.126:42451)
0809 03:35:54.451390 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:54.451394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:54.451595 (+ 201us) server_negotiation.cc:407] Connection header received
0809 03:35:54.451734 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:54.451737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:54.451822 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:54.451912 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:54.452741 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:54.453289 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:54.454211 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:54.454453 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:54.454561 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:54.457310 (+ 2749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:42943: BlockingRecv error: recv got EOF from 127.1.224.66:42943 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 03:35:54 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:54 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:54.672292 16770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:54.666171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:56145)
0809 03:35:54.666482 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:54.666496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:54.666626 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:54.666875 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:54.666877 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:54.667133 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:54.667239 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:54.667245 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:54.668254 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:54.668259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:54.668918 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:54.668928 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:54.669431 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:54.669436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:54.669529 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:54.670061 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:54.670075 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:54.672118 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":123,"threads_started":1}
W20260809 03:35:54.672405 16768 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:54.666341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:56145 (local address 127.1.224.126:42451)
0809 03:35:54.666491 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:54.666495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:54.666607 (+ 112us) server_negotiation.cc:407] Connection header received
0809 03:35:54.666700 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:54.666702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:54.666755 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:54.666847 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:54.667420 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:54.668115 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:54.669049 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:54.669294 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:54.669424 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:54.672271 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:56145: BlockingRecv error: recv got EOF from 127.1.224.67:56145 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 03:35:55 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:55 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:55.226809 16771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:55.219543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:39287)
0809 03:35:55.219982 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:55.219997 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:55.220093 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:55.220469 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:55.220471 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:55.220720 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:55.220828 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:55.220840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:55.222160 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:55.222164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:55.222939 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:55.222952 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:55.223682 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:55.223685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:55.223759 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:55.224499 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:55.224515 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:55.226664 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":88,"threads_started":1}
W20260809 03:35:55.227099 16773 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:55.219682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:39287 (local address 127.1.224.126:42451)
0809 03:35:55.220011 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:55.220018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:55.220044 (+ 26us) server_negotiation.cc:407] Connection header received
0809 03:35:55.220211 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:55.220217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:55.220288 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:55.220413 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:55.220984 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:55.222009 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:55.223064 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:55.223539 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:55.223680 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:55.226900 (+ 3220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:39287: BlockingRecv error: recv got EOF from 127.1.224.65:39287 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":72,"threads_started":1}
Aug 09 03:35:55 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:55 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:55.464354 16774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:55.458319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:37371)
0809 03:35:55.458658 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:55.458673 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:55.458783 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:55.459047 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:55.459049 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:55.459245 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:55.459348 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:55.459353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:55.460192 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:55.460195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:55.460949 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:55.460962 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:55.461523 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:55.461527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:55.461610 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:55.462194 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:55.462206 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:55.464161 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":127,"threads_started":1}
W20260809 03:35:55.464418 16773 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:55.458500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:37371 (local address 127.1.224.126:42451)
0809 03:35:55.458658 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:55.458662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:55.458747 (+ 85us) server_negotiation.cc:407] Connection header received
0809 03:35:55.458873 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:55.458875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:55.458924 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:55.459008 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:55.459453 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:55.460066 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:55.461095 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:55.461391 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:55.461524 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:55.464301 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:37371: BlockingRecv error: recv got EOF from 127.1.224.66:37371 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 03:35:55 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:55 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:55.678830 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:55.673272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:59439)
0809 03:35:55.673578 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:55.673596 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:55.673728 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:55.674067 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:55.674070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:55.674340 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:55.674462 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:55.674468 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:55.675215 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:55.675218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:55.675775 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:55.675782 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:55.676193 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:55.676196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:55.676257 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:55.676810 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:55.676822 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:55.678702 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":91,"threads_started":1}
W20260809 03:35:55.679026 16773 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:55.673344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:59439 (local address 127.1.224.126:42451)
0809 03:35:55.673475 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:55.673480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:55.673684 (+ 204us) server_negotiation.cc:407] Connection header received
0809 03:35:55.673845 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:55.673854 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:55.673910 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:55.674014 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:55.674577 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:55.675120 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:55.675872 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:55.676095 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:55.676171 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:55.678852 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:59439: BlockingRecv error: recv got EOF from 127.1.224.67:59439 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 03:35:56 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:56 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:56.237574 16778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:56.227687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:44501)
0809 03:35:56.227992 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:56.228009 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:56.228128 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:56.228432 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:56.228435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:56.228692 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:56.228871 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:56.228880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:56.229699 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:56.229703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:56.230339 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:56.230349 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:56.230849 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:56.230853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:56.230977 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:56.235252 (+ 4275us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:56.235265 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:56.237364 (+ 2099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":105,"threads_started":1}
W20260809 03:35:56.237641 16780 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:56.227746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:44501 (local address 127.1.224.126:42451)
0809 03:35:56.228107 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:56.228112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:56.228125 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:35:56.228247 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:56.228250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:56.228303 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:56.228400 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:56.229005 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:56.229575 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:56.230464 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:56.230710 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:56.230820 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:56.237509 (+ 6689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:44501: BlockingRecv error: recv got EOF from 127.1.224.65:44501 (error 108)
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":79,"threads_started":1}
Aug 09 03:35:56 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:56 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:56.471386 16781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:56.465370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:53131)
0809 03:35:56.465701 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:56.465716 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:56.465859 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:56.466151 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:56.466154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:56.466466 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:56.466613 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:56.466620 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:56.467435 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:56.467440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:56.468041 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:56.468050 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:56.468541 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:56.468545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:56.468629 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:56.469168 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:56.469179 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:56.471143 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":112,"threads_started":1}
W20260809 03:35:56.471462 16780 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:56.465482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:53131 (local address 127.1.224.126:42451)
0809 03:35:56.465641 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:56.465647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:56.465799 (+ 152us) server_negotiation.cc:407] Connection header received
0809 03:35:56.465977 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:56.465980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:56.466037 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:56.466128 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:56.466740 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:56.467309 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:56.468167 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:56.468405 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:56.468539 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:56.471337 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:53131: BlockingRecv error: recv got EOF from 127.1.224.66:53131 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 03:35:56 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:56 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.67@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:56.685586 16784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:56.679707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.67:44321)
0809 03:35:56.680026 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:56.680040 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:56.680183 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:56.680458 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:56.680461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:56.680719 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:56.680821 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:56.680827 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:56.681630 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:56.681633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:56.682200 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:56.682208 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:56.682734 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:56.682738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:56.682827 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:56.683429 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:56.683443 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:56.685458 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":74,"threads_started":1}
W20260809 03:35:56.685727 16780 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:56.679789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:44321 (local address 127.1.224.126:42451)
0809 03:35:56.679949 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:56.679954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:56.680115 (+ 161us) server_negotiation.cc:407] Connection header received
0809 03:35:56.680287 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:56.680290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:56.680345 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:56.680435 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:56.680944 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:56.681507 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:56.682325 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:56.682615 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:56.682729 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:56.685603 (+ 2874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.67:44321: BlockingRecv error: recv got EOF from 127.1.224.67:44321 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 03:35:57 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:57 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.65@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:57.244968 16785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:57.238787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.65:50553)
0809 03:35:57.239158 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:57.239178 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:57.239301 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:57.239601 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:57.239607 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:57.239886 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:57.240009 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:57.240017 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:57.241048 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:57.241059 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:57.241635 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:57.241643 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:57.242104 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:57.242107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:57.242175 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:57.242803 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:57.242817 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:57.244834 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":117,"threads_started":1}
W20260809 03:35:57.245324 16786 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:57.239010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:50553 (local address 127.1.224.126:42451)
0809 03:35:57.239322 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:57.239327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:57.239341 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:35:57.239388 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:57.239392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:57.239448 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:57.239542 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:57.240173 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:57.240941 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:57.241750 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:57.241990 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:57.242072 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:57.245166 (+ 3094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.65:50553: BlockingRecv error: recv got EOF from 127.1.224.65:50553 (error 108)
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":111,"threads_started":1}
Aug 09 03:35:57 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:35:57 dist-test-slave-jq4t krb5kdc[15213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.224.66@KRBTEST.COM for oryx/127.1.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:35:57.478328 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:57.472633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:42451 (local address 127.1.224.66:43205)
0809 03:35:57.472971 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:57.472986 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:57.473087 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:57.473339 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:57.473341 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:57.473546 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:57.473649 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:57.473654 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:57.474390 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:57.474393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:57.474964 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:57.474972 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:57.475380 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:57.475382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:57.475444 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:57.476014 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:35:57.476026 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:57.478209 (+ 2183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.224.126:42451: Server oryx/127.1.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":136,"threads_started":1}
W20260809 03:35:57.478482 16786 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:57.472687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:43205 (local address 127.1.224.126:42451)
0809 03:35:57.472844 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:57.472848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:57.473058 (+ 210us) server_negotiation.cc:407] Connection header received
0809 03:35:57.473174 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:57.473176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:57.473221 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:57.473298 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:57.473751 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:57.474292 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:57.475067 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:57.475281 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:57.475356 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:57.478351 (+ 2995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.224.66:43205: BlockingRecv error: recv got EOF from 127.1.224.66:43205 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:35:57.572268 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 16081
I20260809 03:35:57.586576 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 16301
I20260809 03:35:57.599155 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 16502
I20260809 03:35:57.607933 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 15978
2026-08-09T03:35:57Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18436 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T03:35:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:57Z Disabled control of system clock
I20260809 03:35:57.648006 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:34225
--webserver_interface=127.1.224.126
--webserver_port=0
--builtin_ntp_servers=127.1.224.84:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:34225
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 03:35:57.767405 16794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:57.767704 16794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:57.767740 16794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:57.771798 16794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:57.771867 16794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:57.771883 16794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:57.771899 16794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:57.771919 16794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:57.776942 16794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:34225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:34225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:57.778184 16794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:57.779699 16794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:57.788815 16801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:57.788944 16799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:57.788898 16794 server_base.cc:1034] running on GCE node
W20260809 03:35:57.788836 16803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:57.789682 16794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:57.790760 16794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:57.791913 16794 hybrid_clock.cc:648] HybridClock initialized: now 1786246557791883 us; error 42 us; skew 500 ppm
I20260809 03:35:57.794008 16794 webserver.cc:460] Webserver started at http://127.1.224.126:39515/ using document root <none> and password file <none>
I20260809 03:35:57.794626 16794 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:57.794691 16794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:57.794947 16794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:57.797207 16794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "73d01edbf3d846fc849f2ca069ae3946"
format_stamp: "Formatted at 2026-08-09 03:35:57 on dist-test-slave-jq4t"
server_key: "6a4dfd62f6d5e3318479665ef6c2d8d1"
server_key_iv: "ca401a40c80212d670481c5f18d24112"
server_key_version: "encryptionkey@0"
I20260809 03:35:57.797791 16794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "73d01edbf3d846fc849f2ca069ae3946"
format_stamp: "Formatted at 2026-08-09 03:35:57 on dist-test-slave-jq4t"
server_key: "6a4dfd62f6d5e3318479665ef6c2d8d1"
server_key_iv: "ca401a40c80212d670481c5f18d24112"
server_key_version: "encryptionkey@0"
I20260809 03:35:57.801589 16794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:35:57.804211 16813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:57.805559 16794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:57.805665 16794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "73d01edbf3d846fc849f2ca069ae3946"
format_stamp: "Formatted at 2026-08-09 03:35:57 on dist-test-slave-jq4t"
server_key: "6a4dfd62f6d5e3318479665ef6c2d8d1"
server_key_iv: "ca401a40c80212d670481c5f18d24112"
server_key_version: "encryptionkey@0"
I20260809 03:35:57.805756 16794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:57.816872 16794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:57.817528 16794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:57.817684 16794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:57.824990 16794 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:34225
I20260809 03:35:57.825008 16890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:34225 every 8 connection(s)
I20260809 03:35:57.825994 16794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:57.829648 16891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:57.834663 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 16794
I20260809 03:35:57.834800 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:57.835129 1921 external_mini_cluster.cc:1428] Setting key 4067d748dcffc91bae534c74dce8f2fb
I20260809 03:35:57.836444 16891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946: Bootstrap starting.
I20260809 03:35:57.839166 16891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:57.839977 16891 log.cc:824] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:57.842033 16891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946: No bootstrap required, opened a new log
I20260809 03:35:57.845355 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:57.836903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.1.224.126:34225)
0809 03:35:57.837676 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:57.837694 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:57.837733 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:35:57.838431 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:57.838451 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:57.838896 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:57.839277 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:57.839752 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:57.840877 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:57.841815 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:57.842371 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:57.842553 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:57.843499 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:57.843504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:57.843517 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:57.843922 (+ 405us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:57.843981 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:57.844001 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:57.844116 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:57.844278 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 03:35:57.844605 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":152,"threads_started":1}
I20260809 03:35:57.846005 16891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d01edbf3d846fc849f2ca069ae3946" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 34225 } }
I20260809 03:35:57.846259 16891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:57.846355 16891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73d01edbf3d846fc849f2ca069ae3946, State: Initialized, Role: FOLLOWER
I20260809 03:35:57.846838 16891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d01edbf3d846fc849f2ca069ae3946" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 34225 } }
I20260809 03:35:57.847003 16891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:57.847091 16891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:57.847227 16891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:57.848343 16891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d01edbf3d846fc849f2ca069ae3946" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 34225 } }
I20260809 03:35:57.848730 16891 leader_election.cc:302] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73d01edbf3d846fc849f2ca069ae3946; no voters:
I20260809 03:35:57.849057 16891 leader_election.cc:288] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:57.849205 16901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:57.849465 16901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [term 1 LEADER]: Becoming Leader. State: Replica: 73d01edbf3d846fc849f2ca069ae3946, State: Running, Role: LEADER
I20260809 03:35:57.849875 16901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d01edbf3d846fc849f2ca069ae3946" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 34225 } }
I20260809 03:35:57.850410 16891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:57.851684 16903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73d01edbf3d846fc849f2ca069ae3946. Latest consensus state: current_term: 1 leader_uuid: "73d01edbf3d846fc849f2ca069ae3946" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d01edbf3d846fc849f2ca069ae3946" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 34225 } } }
I20260809 03:35:57.851658 16902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73d01edbf3d846fc849f2ca069ae3946" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d01edbf3d846fc849f2ca069ae3946" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 34225 } } }
I20260809 03:35:57.851850 16902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:57.851850 16903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946 [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:57.852300 16912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:57.855358 16912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:57.860932 16912 catalog_manager.cc:1347] Generated new cluster ID: 1e45931480594be68dc744352ef578b2
I20260809 03:35:57.861014 16912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:57.870146 16912 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:57.871441 16912 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:57.879053 16912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73d01edbf3d846fc849f2ca069ae3946: Generated new TSK 0
I20260809 03:35:57.879899 16912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:35:57.890810 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:34225
--builtin_ntp_servers=127.1.224.84:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 03:35:58.011152 16932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:58.011437 16932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:58.011476 16932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:58.015626 16932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:58.015699 16932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:58.015753 16932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:58.020797 16932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:34225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:58.022174 16932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:58.023706 16932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:58.034243 16948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:58.034289 16943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:58.034412 16932 server_base.cc:1034] running on GCE node
W20260809 03:35:58.034646 16944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:58.035176 16932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:58.035921 16932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:58.037098 16932 hybrid_clock.cc:648] HybridClock initialized: now 1786246558037084 us; error 31 us; skew 500 ppm
I20260809 03:35:58.039219 16932 webserver.cc:460] Webserver started at http://127.1.224.65:34783/ using document root <none> and password file <none>
I20260809 03:35:58.039863 16932 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:58.039933 16932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:58.040235 16932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:58.042505 16932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "e6f540a96b4246519e785e97b853a44c"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "38511d9548d9835e37c79217161c459d"
server_key_iv: "10b69abf97d2c2047626183786d842bc"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.043107 16932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "e6f540a96b4246519e785e97b853a44c"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "38511d9548d9835e37c79217161c459d"
server_key_iv: "10b69abf97d2c2047626183786d842bc"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.047282 16932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 03:35:58.050352 16955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:58.051841 16932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:35:58.051971 16932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "e6f540a96b4246519e785e97b853a44c"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "38511d9548d9835e37c79217161c459d"
server_key_iv: "10b69abf97d2c2047626183786d842bc"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.052104 16932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:58.070920 16932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:58.071687 16932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:58.071900 16932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:58.072592 16932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:58.073606 16932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:58.073657 16932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:58.073727 16932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:58.073758 16932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:58.083765 16932 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:45657
I20260809 03:35:58.083799 17120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:45657 every 8 connection(s)
I20260809 03:35:58.084813 16932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:58.088979 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 16932
I20260809 03:35:58.089087 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:58.089411 1921 external_mini_cluster.cc:1428] Setting key 127b37bf62f3a9741dedb83d3c366fb7
I20260809 03:35:58.091658 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:34225
--builtin_ntp_servers=127.1.224.84:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 03:35:58.095516 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:58.086731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:59743 (local address 127.1.224.126:34225)
0809 03:35:58.086895 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:58.086900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:58.088074 (+ 1174us) server_negotiation.cc:407] Connection header received
0809 03:35:58.089701 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:58.089705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:58.089765 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:58.089876 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:58.090883 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:58.091660 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:58.092931 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:58.093340 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:58.093454 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:58.094843 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:58.094850 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:58.094856 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:58.094874 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:58.094901 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:58.094914 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:58.094965 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:58.095204 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 03:35:58.095279 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:35:58.097201 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:58.087227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:34225 (local address 127.1.224.65:59743)
0809 03:35:58.087902 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:58.087945 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:58.089436 (+ 1491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:58.090084 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:58.090094 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:58.090193 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:58.090700 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:58.090714 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:58.091826 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:58.091832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:58.092800 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:58.092810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:58.093532 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:58.093539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:58.093776 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:58.094461 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:58.094464 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:58.094539 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:58.094654 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:58.095038 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:58.095050 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:58.095058 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 03:35:58.095695 (+ 637us) client_negotiation.cc:240] Negotiation successful
0809 03:35:58.096010 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":117,"threads_started":1}
I20260809 03:35:58.098531 17121 heartbeater.cc:347] Connected to a master server at 127.1.224.126:34225
I20260809 03:35:58.098798 17121 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:58.099364 17121 heartbeater.cc:511] Master 127.1.224.126:34225 requested a full tablet report, sending...
I20260809 03:35:58.100972 16840 ts_manager.cc:194] Registered new tserver with Master: e6f540a96b4246519e785e97b853a44c (127.1.224.65:45657)
I20260809 03:35:58.102100 16840 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.224.65:59743
W20260809 03:35:58.210126 17130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:58.210467 17130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:58.210544 17130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:58.214682 17130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:58.214788 17130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:58.214887 17130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:58.219923 17130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:34225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:58.221166 17130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:58.222766 17130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:58.233198 17140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:58.233198 17139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:58.233209 17142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:58.233533 17130 server_base.cc:1034] running on GCE node
I20260809 03:35:58.234105 17130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:58.234803 17130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:58.235996 17130 hybrid_clock.cc:648] HybridClock initialized: now 1786246558235966 us; error 41 us; skew 500 ppm
I20260809 03:35:58.238189 17130 webserver.cc:460] Webserver started at http://127.1.224.66:38737/ using document root <none> and password file <none>
I20260809 03:35:58.238900 17130 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:58.238999 17130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:58.239250 17130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:58.241566 17130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "6ed455d1c22741c791f70e5c08027b9a"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "ec6099edf6478e69d10b8ea1d97f7c87"
server_key_iv: "69974837b4bd4cc15753c09a8890b9b7"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.242169 17130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "6ed455d1c22741c791f70e5c08027b9a"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "ec6099edf6478e69d10b8ea1d97f7c87"
server_key_iv: "69974837b4bd4cc15753c09a8890b9b7"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.246206 17130 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 03:35:58.249210 17151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:58.250623 17130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:35:58.250761 17130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "6ed455d1c22741c791f70e5c08027b9a"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "ec6099edf6478e69d10b8ea1d97f7c87"
server_key_iv: "69974837b4bd4cc15753c09a8890b9b7"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.250898 17130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:58.277223 17130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:58.277962 17130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:58.278173 17130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:58.278806 17130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:58.279793 17130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:58.279868 17130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:58.279944 17130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:58.279989 17130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:58.288882 17130 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:43669
I20260809 03:35:58.288916 17328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:43669 every 8 connection(s)
I20260809 03:35:58.289849 17130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:58.290366 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 17130
I20260809 03:35:58.290482 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:58.290861 1921 external_mini_cluster.cc:1428] Setting key c64ab3c7dc6da443fb21a48bf35556ad
I20260809 03:35:58.293143 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:34225
--builtin_ntp_servers=127.1.224.84:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 03:35:58.301115 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:58.291949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:46693 (local address 127.1.224.126:34225)
0809 03:35:58.292134 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:58.292139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:58.293516 (+ 1377us) server_negotiation.cc:407] Connection header received
0809 03:35:58.295066 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:58.295069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:58.295116 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:58.295183 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:58.296429 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:58.296994 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:58.298876 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:58.299094 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:58.299173 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:58.300611 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:58.300614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:58.300616 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:58.300629 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:58.300649 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:58.300658 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:58.300733 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:58.300933 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 03:35:58.300986 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 03:35:58.302425 17336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:58.292555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:34225 (local address 127.1.224.66:46693)
0809 03:35:58.293366 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:58.293420 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:58.294818 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:58.295396 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:58.295406 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:58.295520 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:58.296275 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:58.296298 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:58.297136 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:58.297142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:58.298743 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:58.298760 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:58.299270 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:58.299278 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:58.299525 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:58.300236 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:58.300241 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:58.300302 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:58.300424 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:58.300758 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:58.300771 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:58.300779 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 03:35:58.300929 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 03:35:58.301280 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":149,"threads_started":1}
I20260809 03:35:58.303701 17332 heartbeater.cc:347] Connected to a master server at 127.1.224.126:34225
I20260809 03:35:58.303938 17332 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:58.304436 17332 heartbeater.cc:511] Master 127.1.224.126:34225 requested a full tablet report, sending...
I20260809 03:35:58.305550 16840 ts_manager.cc:194] Registered new tserver with Master: 6ed455d1c22741c791f70e5c08027b9a (127.1.224.66:43669)
I20260809 03:35:58.306097 16840 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.224.66:46693
W20260809 03:35:58.415323 17337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:58.415666 17337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:58.415747 17337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:58.420014 17337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:58.420118 17337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:58.420213 17337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:58.425261 17337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:34225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:58.426505 17337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:58.427997 17337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:58.438629 17343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:58.438764 17337 server_base.cc:1034] running on GCE node
W20260809 03:35:58.438629 17342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:58.438654 17345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:58.439450 17337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:58.440155 17337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:58.441324 17337 hybrid_clock.cc:648] HybridClock initialized: now 1786246558441285 us; error 51 us; skew 500 ppm
I20260809 03:35:58.443260 17337 webserver.cc:460] Webserver started at http://127.1.224.67:36111/ using document root <none> and password file <none>
I20260809 03:35:58.443845 17337 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:58.443909 17337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:58.444080 17337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:58.446259 17337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "037e0ade1fae4f88be53f634b1a1e6af"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "728fe969ddbfc878e7293b83d28f4f86"
server_key_iv: "27b3638095fc10d81f398e1015ec9f4d"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.446795 17337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "037e0ade1fae4f88be53f634b1a1e6af"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "728fe969ddbfc878e7293b83d28f4f86"
server_key_iv: "27b3638095fc10d81f398e1015ec9f4d"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.450965 17337 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 03:35:58.453876 17356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:58.455320 17337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 03:35:58.455441 17337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "037e0ade1fae4f88be53f634b1a1e6af"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "728fe969ddbfc878e7293b83d28f4f86"
server_key_iv: "27b3638095fc10d81f398e1015ec9f4d"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.455547 17337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:58.468124 17337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:58.469034 17337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:58.469218 17337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:58.470048 17337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:58.471390 17337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:58.471441 17337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:58.471518 17337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:58.471550 17337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:58.481408 17337 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:41959
I20260809 03:35:58.481495 17524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:41959 every 8 connection(s)
I20260809 03:35:58.482520 17337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:58.491487 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 17337
I20260809 03:35:58.491633 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:58.491952 1921 external_mini_cluster.cc:1428] Setting key 58a5c343f795e252cd0311a9f8a565ac
I20260809 03:35:58.494091 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:58.484773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:56013 (local address 127.1.224.126:34225)
0809 03:35:58.484951 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:58.484955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:58.486954 (+ 1999us) server_negotiation.cc:407] Connection header received
0809 03:35:58.488464 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:58.488467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:58.488560 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:58.488675 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:58.489646 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:58.490209 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:58.491652 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:35:58.491920 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:35:58.492069 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:58.493563 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:58.493569 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:58.493576 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:58.493590 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:58.493610 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:58.493618 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:58.493698 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:58.493874 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 03:35:58.493941 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:35:58.495124 17527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:58.485291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:34225 (local address 127.1.224.67:56013)
0809 03:35:58.486775 (+ 1484us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:58.486822 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:58.488223 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:58.488870 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:58.488878 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:58.488964 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:58.489491 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:58.489504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:58.490366 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:58.490369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:58.491442 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:35:58.491457 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:35:58.492111 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:35:58.492115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:35:58.492343 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:35:58.493268 (+ 925us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:58.493270 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:58.493311 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:58.493391 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:58.493713 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:58.493724 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:58.493730 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:35:58.493870 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 03:35:58.494170 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"thread_start_us":840,"threads_started":1}
I20260809 03:35:58.496953 17525 heartbeater.cc:347] Connected to a master server at 127.1.224.126:34225
I20260809 03:35:58.497208 17525 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:58.497769 17525 heartbeater.cc:511] Master 127.1.224.126:34225 requested a full tablet report, sending...
I20260809 03:35:58.498811 16840 ts_manager.cc:194] Registered new tserver with Master: 037e0ade1fae4f88be53f634b1a1e6af (127.1.224.67:41959)
I20260809 03:35:58.499411 16840 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.224.67:56013
I20260809 03:35:58.506150 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 03:35:58.509261 16898 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:58.508141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38814 (local address 127.1.224.126:34225)
0809 03:35:58.508259 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:58.508263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:58.508528 (+ 265us) server_negotiation.cc:407] Connection header received
0809 03:35:58.508619 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:58.508622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:58.508667 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:58.508737 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:58.509111 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38814: BlockingRecv error: recv got EOF from 127.0.0.1:38814 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:35:58.509357 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 16932
I20260809 03:35:58.537025 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 17130
I20260809 03:35:58.547601 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 17337
I20260809 03:35:58.563941 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 16794
2026-08-09T03:35:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (954 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T03:35:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:35:58Z Disabled control of system clock
I20260809 03:35:58.602421 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:39013
--webserver_interface=127.1.224.126
--webserver_port=0
--builtin_ntp_servers=127.1.224.84:38719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:39013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 03:35:58.720191 17542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:58.720528 17542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:58.720587 17542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:58.724696 17542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:35:58.724758 17542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:58.724774 17542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:58.724792 17542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:35:58.724807 17542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:35:58.729724 17542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:38719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:39013
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:39013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:58.730901 17542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:58.732290 17542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:58.741070 17555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:58.741040 17549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:58.741206 17542 server_base.cc:1034] running on GCE node
W20260809 03:35:58.741408 17550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:58.742030 17542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:58.743182 17542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:58.744350 17542 hybrid_clock.cc:648] HybridClock initialized: now 1786246558744330 us; error 32 us; skew 500 ppm
I20260809 03:35:58.746268 17542 webserver.cc:460] Webserver started at http://127.1.224.126:36093/ using document root <none> and password file <none>
I20260809 03:35:58.746871 17542 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:58.746934 17542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:58.747207 17542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:58.749392 17542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "bfedc7dc5a174fec8e00921c3f953f5d"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "65172b9adf00affb024492692bfe80bd"
server_key_iv: "6ca8bf6f41155f198a5ffc0e1c924145"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.749984 17542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "bfedc7dc5a174fec8e00921c3f953f5d"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "65172b9adf00affb024492692bfe80bd"
server_key_iv: "6ca8bf6f41155f198a5ffc0e1c924145"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.753813 17542 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:35:58.756240 17565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:58.757498 17542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:35:58.757663 17542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "bfedc7dc5a174fec8e00921c3f953f5d"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "65172b9adf00affb024492692bfe80bd"
server_key_iv: "6ca8bf6f41155f198a5ffc0e1c924145"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.757771 17542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:58.762557 17542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:58.763204 17542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:58.763401 17542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:58.770812 17542 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:39013
I20260809 03:35:58.770818 17637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:39013 every 8 connection(s)
I20260809 03:35:58.771790 17542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:35:58.775112 17638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:35:58.778677 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 17542
I20260809 03:35:58.778803 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:35:58.779107 1921 external_mini_cluster.cc:1428] Setting key 4f3d01b0f52a85d1286eb84301d4aa97
I20260809 03:35:58.781435 17638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d: Bootstrap starting.
I20260809 03:35:58.783849 17638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d: Neither blocks nor log segments found. Creating new log.
I20260809 03:35:58.784794 17638 log.cc:824] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:35:58.785214 17644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:58.780768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.1.224.126:39013)
0809 03:35:58.781444 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:58.781459 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:58.781495 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:35:58.782196 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:58.782214 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:58.782569 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:58.782917 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:58.783766 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:58.783771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:58.783784 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:58.783796 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:58.783852 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:58.783868 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:58.783988 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:58.784110 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 03:35:58.784360 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":144,"threads_started":1}
I20260809 03:35:58.787042 17638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d: No bootstrap required, opened a new log
I20260809 03:35:58.790957 17638 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfedc7dc5a174fec8e00921c3f953f5d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39013 } }
I20260809 03:35:58.791262 17638 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:35:58.791343 17638 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfedc7dc5a174fec8e00921c3f953f5d, State: Initialized, Role: FOLLOWER
I20260809 03:35:58.791829 17638 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfedc7dc5a174fec8e00921c3f953f5d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39013 } }
I20260809 03:35:58.791965 17638 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:35:58.792013 17638 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:35:58.792088 17638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:35:58.793097 17638 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfedc7dc5a174fec8e00921c3f953f5d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39013 } }
I20260809 03:35:58.793398 17638 leader_election.cc:302] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfedc7dc5a174fec8e00921c3f953f5d; no voters:
I20260809 03:35:58.793668 17638 leader_election.cc:288] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:35:58.793797 17647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:35:58.794080 17647 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [term 1 LEADER]: Becoming Leader. State: Replica: bfedc7dc5a174fec8e00921c3f953f5d, State: Running, Role: LEADER
I20260809 03:35:58.794407 17647 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfedc7dc5a174fec8e00921c3f953f5d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39013 } }
I20260809 03:35:58.795008 17638 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:35:58.796010 17650 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfedc7dc5a174fec8e00921c3f953f5d. Latest consensus state: current_term: 1 leader_uuid: "bfedc7dc5a174fec8e00921c3f953f5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfedc7dc5a174fec8e00921c3f953f5d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39013 } } }
I20260809 03:35:58.796147 17650 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:58.796602 17657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:35:58.796550 17649 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfedc7dc5a174fec8e00921c3f953f5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfedc7dc5a174fec8e00921c3f953f5d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39013 } } }
I20260809 03:35:58.796721 17649 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d [sys.catalog]: This master's current role is: LEADER
I20260809 03:35:58.799960 17657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:35:58.806206 17657 catalog_manager.cc:1347] Generated new cluster ID: baa50e00d3f7493cb66abe5b8692234f
I20260809 03:35:58.806288 17657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:35:58.812157 17657 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:35:58.812465 17657 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:35:58.817818 17657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfedc7dc5a174fec8e00921c3f953f5d: Generated new TSK 0
I20260809 03:35:58.818535 17657 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:35:58.826850 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39013
--builtin_ntp_servers=127.1.224.84:38719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 03:35:58.946516 17668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:58.946775 17668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:58.946847 17668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:58.950847 17668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:58.950901 17668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:58.951002 17668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:35:58.955744 17668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:38719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:58.956979 17668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:58.958428 17668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:58.968688 17678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:58.968668 17681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:58.968665 17676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:58.969117 17668 server_base.cc:1034] running on GCE node
I20260809 03:35:58.969521 17668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:58.970149 17668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:58.971323 17668 hybrid_clock.cc:648] HybridClock initialized: now 1786246558971306 us; error 28 us; skew 500 ppm
I20260809 03:35:58.973305 17668 webserver.cc:460] Webserver started at http://127.1.224.65:37815/ using document root <none> and password file <none>
I20260809 03:35:58.973908 17668 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:58.974004 17668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:58.974253 17668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:58.976353 17668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "754afba1426d488188b384a1562c28f6"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "7a1e403068aae0ea988943df99019e48"
server_key_iv: "ce4aaedb5fc6dcdeafe17e3b1370ecee"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.977000 17668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "754afba1426d488188b384a1562c28f6"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "7a1e403068aae0ea988943df99019e48"
server_key_iv: "ce4aaedb5fc6dcdeafe17e3b1370ecee"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.980728 17668 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:35:58.983464 17691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:58.984884 17668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:35:58.985028 17668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "754afba1426d488188b384a1562c28f6"
format_stamp: "Formatted at 2026-08-09 03:35:58 on dist-test-slave-jq4t"
server_key: "7a1e403068aae0ea988943df99019e48"
server_key_iv: "ce4aaedb5fc6dcdeafe17e3b1370ecee"
server_key_version: "encryptionkey@0"
I20260809 03:35:58.985152 17668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:58.989831 17668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:58.990471 17668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:58.990669 17668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:58.991261 17668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:58.992266 17668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:58.992350 17668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:58.992442 17668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:58.992513 17668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 03:35:59.001660 17668 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:34673
I20260809 03:35:59.001693 17851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:34673 every 8 connection(s)
I20260809 03:35:59.002574 17668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:35:59.004465 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 17668
I20260809 03:35:59.004616 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:35:59.004942 1921 external_mini_cluster.cc:1428] Setting key 50346a1a4280cac0b2a369f5b32bb462
I20260809 03:35:59.007161 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39013
--builtin_ntp_servers=127.1.224.84:38719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 03:35:59.009456 17644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:59.004385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:47795 (local address 127.1.224.126:39013)
0809 03:35:59.004580 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:59.004585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:59.005988 (+ 1403us) server_negotiation.cc:407] Connection header received
0809 03:35:59.007586 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:59.007592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:59.007637 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:59.007732 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:59.008973 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:59.008980 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:59.008986 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:59.009003 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:59.009034 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:59.009046 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:59.009132 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:59.009308 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 03:35:59.009348 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:35:59.010648 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:59.004989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39013 (local address 127.1.224.65:47795)
0809 03:35:59.005819 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:59.005875 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:59.007327 (+ 1452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:59.007890 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:59.007896 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:59.007990 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:59.008657 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:59.008659 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:59.008698 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:59.008767 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:59.009147 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:59.009159 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:59.009167 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 03:35:59.009350 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 03:35:59.009604 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":152,"threads_started":1}
I20260809 03:35:59.011879 17852 heartbeater.cc:347] Connected to a master server at 127.1.224.126:39013
I20260809 03:35:59.012130 17852 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:59.012615 17852 heartbeater.cc:511] Master 127.1.224.126:39013 requested a full tablet report, sending...
I20260809 03:35:59.014190 17585 ts_manager.cc:194] Registered new tserver with Master: 754afba1426d488188b384a1562c28f6 (127.1.224.65:34673)
W20260809 03:35:59.126847 17855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:59.127166 17855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:59.127240 17855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:59.131371 17855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:59.131464 17855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:59.131569 17855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:35:59.136617 17855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:38719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:59.137871 17855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:59.139394 17855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:59.149564 17869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:59.149573 17864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:59.149540 17865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:59.149921 17855 server_base.cc:1034] running on GCE node
I20260809 03:35:59.150426 17855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:59.151158 17855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:59.152365 17855 hybrid_clock.cc:648] HybridClock initialized: now 1786246559152346 us; error 34 us; skew 500 ppm
I20260809 03:35:59.154412 17855 webserver.cc:460] Webserver started at http://127.1.224.66:43465/ using document root <none> and password file <none>
I20260809 03:35:59.155076 17855 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:59.155169 17855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:59.155433 17855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:59.157704 17855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "5ab1057fa2be42a4ba3db993f1b31582"
format_stamp: "Formatted at 2026-08-09 03:35:59 on dist-test-slave-jq4t"
server_key: "e9700c62ea1e43755ed7053a3e9832a7"
server_key_iv: "e04b736849427645af43a4ebe1075890"
server_key_version: "encryptionkey@0"
I20260809 03:35:59.158294 17855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "5ab1057fa2be42a4ba3db993f1b31582"
format_stamp: "Formatted at 2026-08-09 03:35:59 on dist-test-slave-jq4t"
server_key: "e9700c62ea1e43755ed7053a3e9832a7"
server_key_iv: "e04b736849427645af43a4ebe1075890"
server_key_version: "encryptionkey@0"
I20260809 03:35:59.162539 17855 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:35:59.165273 17875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:59.166556 17855 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:35:59.166651 17855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "5ab1057fa2be42a4ba3db993f1b31582"
format_stamp: "Formatted at 2026-08-09 03:35:59 on dist-test-slave-jq4t"
server_key: "e9700c62ea1e43755ed7053a3e9832a7"
server_key_iv: "e04b736849427645af43a4ebe1075890"
server_key_version: "encryptionkey@0"
I20260809 03:35:59.166731 17855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:59.171388 17855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:59.171950 17855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:59.172084 17855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:59.172664 17855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:59.173554 17855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:59.173604 17855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:59.173640 17855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:59.173660 17855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:59.182590 17855 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:39897
I20260809 03:35:59.182611 18058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:39897 every 8 connection(s)
I20260809 03:35:59.183776 17855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:35:59.185652 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 17855
I20260809 03:35:59.185789 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:35:59.186175 1921 external_mini_cluster.cc:1428] Setting key c35a2648c034695f74fd2f1014b2188d
I20260809 03:35:59.188580 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39013
--builtin_ntp_servers=127.1.224.84:38719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 03:35:59.191334 17644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:59.186107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:53339 (local address 127.1.224.126:39013)
0809 03:35:59.186265 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:59.186271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:59.187756 (+ 1485us) server_negotiation.cc:407] Connection header received
0809 03:35:59.189446 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:59.189449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:59.189498 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:59.189604 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:59.190821 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:59.190825 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:59.190840 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:59.190854 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:59.190882 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:59.190891 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:59.190998 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:59.191143 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 03:35:59.191191 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:35:59.192329 18063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:59.186673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39013 (local address 127.1.224.66:53339)
0809 03:35:59.187591 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:59.187650 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:59.189177 (+ 1527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:59.189792 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:59.189799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:59.189890 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:59.190551 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:59.190553 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:59.190596 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:59.190668 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:59.190978 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:59.190988 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:59.190994 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:35:59.191192 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 03:35:59.191458 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":163,"threads_started":1}
I20260809 03:35:59.193575 18061 heartbeater.cc:347] Connected to a master server at 127.1.224.126:39013
I20260809 03:35:59.193840 18061 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:59.194396 18061 heartbeater.cc:511] Master 127.1.224.126:39013 requested a full tablet report, sending...
I20260809 03:35:59.195645 17585 ts_manager.cc:194] Registered new tserver with Master: 5ab1057fa2be42a4ba3db993f1b31582 (127.1.224.66:39897)
W20260809 03:35:59.312268 18064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:35:59.312641 18064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:35:59.312722 18064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:35:59.316987 18064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:35:59.317052 18064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:35:59.317142 18064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:35:59.322412 18064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:38719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:39013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:35:59.323688 18064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:35:59.325378 18064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:35:59.335738 18074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:59.335841 18064 server_base.cc:1034] running on GCE node
W20260809 03:35:59.335731 18077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:35:59.335733 18072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:35:59.336467 18064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:35:59.337268 18064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:35:59.338464 18064 hybrid_clock.cc:648] HybridClock initialized: now 1786246559338451 us; error 26 us; skew 500 ppm
I20260809 03:35:59.340461 18064 webserver.cc:460] Webserver started at http://127.1.224.67:37559/ using document root <none> and password file <none>
I20260809 03:35:59.341164 18064 fs_manager.cc:359] Metadata directory not provided
I20260809 03:35:59.341266 18064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:35:59.341521 18064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:35:59.343678 18064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "cdb85020afcc49928956451f7f6f792c"
format_stamp: "Formatted at 2026-08-09 03:35:59 on dist-test-slave-jq4t"
server_key: "f6954f79b4067d40be306a6c1bba4cc6"
server_key_iv: "11aead0395068e310959496ca2118332"
server_key_version: "encryptionkey@0"
I20260809 03:35:59.344277 18064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "cdb85020afcc49928956451f7f6f792c"
format_stamp: "Formatted at 2026-08-09 03:35:59 on dist-test-slave-jq4t"
server_key: "f6954f79b4067d40be306a6c1bba4cc6"
server_key_iv: "11aead0395068e310959496ca2118332"
server_key_version: "encryptionkey@0"
I20260809 03:35:59.348156 18064 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260809 03:35:59.350811 18085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:59.352078 18064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:35:59.352224 18064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "cdb85020afcc49928956451f7f6f792c"
format_stamp: "Formatted at 2026-08-09 03:35:59 on dist-test-slave-jq4t"
server_key: "f6954f79b4067d40be306a6c1bba4cc6"
server_key_iv: "11aead0395068e310959496ca2118332"
server_key_version: "encryptionkey@0"
I20260809 03:35:59.352346 18064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:35:59.357288 18064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:35:59.357978 18064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:35:59.358189 18064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:35:59.358781 18064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:35:59.359761 18064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:35:59.359836 18064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:59.359902 18064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:35:59.359956 18064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:35:59.369433 18064 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:35209
I20260809 03:35:59.369457 18253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:35209 every 8 connection(s)
I20260809 03:35:59.370510 18064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:35:59.377477 17644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:59.372611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:56775 (local address 127.1.224.126:39013)
0809 03:35:59.372790 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:59.372795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:59.374131 (+ 1336us) server_negotiation.cc:407] Connection header received
0809 03:35:59.375520 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:59.375523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:59.375567 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:59.375651 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:59.377006 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:35:59.377009 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:35:59.377012 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:35:59.377026 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:35:59.377051 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:35:59.377060 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:35:59.377163 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 03:35:59.377303 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 03:35:59.377353 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:35:59.377913 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 18064
I20260809 03:35:59.378013 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:35:59.378356 1921 external_mini_cluster.cc:1428] Setting key dcbf65539e2c576a941a4046319066ec
I20260809 03:35:59.378593 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:35:59.373163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:39013 (local address 127.1.224.67:56775)
0809 03:35:59.373943 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 03:35:59.373987 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:35:59.375301 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:35:59.375841 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:35:59.375848 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:35:59.375943 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:35:59.376686 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:35:59.376687 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:35:59.376734 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:35:59.376815 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:35:59.377157 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:35:59.377164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:35:59.377177 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 03:35:59.377324 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 03:35:59.377583 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":116,"threads_started":1}
I20260809 03:35:59.380059 18254 heartbeater.cc:347] Connected to a master server at 127.1.224.126:39013
I20260809 03:35:59.380383 18254 heartbeater.cc:464] Registering TS with master...
I20260809 03:35:59.381007 18254 heartbeater.cc:511] Master 127.1.224.126:39013 requested a full tablet report, sending...
I20260809 03:35:59.382521 17585 ts_manager.cc:194] Registered new tserver with Master: cdb85020afcc49928956451f7f6f792c (127.1.224.67:35209)
I20260809 03:35:59.394860 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:35:59.398113 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 17668
W20260809 03:35:59.398157 17644 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:35:59.397131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48822 (local address 127.1.224.126:39013)
0809 03:35:59.397296 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:35:59.397301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:35:59.397426 (+ 125us) server_negotiation.cc:407] Connection header received
0809 03:35:59.397540 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:35:59.397542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:35:59.397580 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:35:59.397650 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:35:59.397961 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48822: BlockingRecv error: recv got EOF from 127.0.0.1:48822 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 03:35:59.413717 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 17855
I20260809 03:35:59.429489 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 18064
I20260809 03:35:59.450953 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 17542
2026-08-09T03:35:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (886 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:35:59 dist-test-slave-jq4t krb5kdc[18272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:35:59 dist-test-slave-jq4t krb5kdc[18272](info): set up 2 sockets
Aug 09 03:35:59 dist-test-slave-jq4t krb5kdc[18272](info): commencing operation
krb5kdc: starting...
W20260809 03:36:00.712946 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:36:00 dist-test-slave-jq4t krb5kdc[18272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:36:00.845665 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:43485
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:43485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:00.964953 18290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:00.965245 18290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:00.965286 18290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:00.969282 18290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:00.969353 18290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:00.969373 18290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:00.969390 18290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:00.969406 18290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:00.974915 18290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:43485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:43485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:36:00.976058 18290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:00.977125 18290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:00.986318 18298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:00.986318 18302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:00.986336 18297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:00.986510 18290 server_base.cc:1034] running on GCE node
I20260809 03:36:00.987401 18290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:00.988724 18290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:00.989931 18290 hybrid_clock.cc:648] HybridClock initialized: now 1786246560989916 us; error 29 us; skew 500 ppm
Aug 09 03:36:00 dist-test-slave-jq4t krb5kdc[18272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246560, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:00.992956 18290 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:36:00.993866 18290 webserver.cc:460] Webserver started at http://127.1.224.126:46739/ using document root <none> and password file <none>
I20260809 03:36:00.994432 18290 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:00.994494 18290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:00.994671 18290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:00.996855 18290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "566b0293b66f485383de9917f2f22a7e"
format_stamp: "Formatted at 2026-08-09 03:36:00 on dist-test-slave-jq4t"
server_key: "11981f4e326e030f5f12b6dd806cce48"
server_key_iv: "77c4eaa48b5d6124a1bdfa9eff5f1f51"
server_key_version: "encryptionkey@0"
I20260809 03:36:00.997413 18290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "566b0293b66f485383de9917f2f22a7e"
format_stamp: "Formatted at 2026-08-09 03:36:00 on dist-test-slave-jq4t"
server_key: "11981f4e326e030f5f12b6dd806cce48"
server_key_iv: "77c4eaa48b5d6124a1bdfa9eff5f1f51"
server_key_version: "encryptionkey@0"
I20260809 03:36:01.001312 18290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:36:01.003775 18312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:01.005095 18290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:36:01.005245 18290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "566b0293b66f485383de9917f2f22a7e"
format_stamp: "Formatted at 2026-08-09 03:36:00 on dist-test-slave-jq4t"
server_key: "11981f4e326e030f5f12b6dd806cce48"
server_key_iv: "77c4eaa48b5d6124a1bdfa9eff5f1f51"
server_key_version: "encryptionkey@0"
I20260809 03:36:01.005350 18290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:01.015666 18290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:01.018960 18290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:01.019193 18290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:01.026751 18290 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:43485
I20260809 03:36:01.027057 18385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:43485 every 8 connection(s)
I20260809 03:36:01.027940 18290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:01.031073 18386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:01.033105 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 18290
I20260809 03:36:01.033212 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:36:01.033528 1921 external_mini_cluster.cc:1428] Setting key 3bb235641844292575389cf7aa46e462
I20260809 03:36:01.037923 18386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e: Bootstrap starting.
I20260809 03:36:01.040212 18386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:01.041211 18386 log.cc:824] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:36:01 dist-test-slave-jq4t krb5kdc[18272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:36:01.043363 18386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e: No bootstrap required, opened a new log
I20260809 03:36:01.047221 18386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566b0293b66f485383de9917f2f22a7e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43485 } }
I20260809 03:36:01.047531 18386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:01.047618 18386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 566b0293b66f485383de9917f2f22a7e, State: Initialized, Role: FOLLOWER
I20260809 03:36:01.048084 18386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566b0293b66f485383de9917f2f22a7e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43485 } }
I20260809 03:36:01.048213 18386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:01.048305 18386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:01.048410 18386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:01.048439 18390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.035068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39622 (local address 127.1.224.126:43485)
0809 03:36:01.035558 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:01.035578 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:01.035614 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:36:01.036265 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:01.036283 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:01.036714 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:01.037067 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:01.037907 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.038958 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.039837 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.040343 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.040577 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.043329 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:01.043334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:01.043348 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:01.043375 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:01.045174 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:01.045649 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:01.045655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:01.045660 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:01.045775 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:01.046109 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:01.046113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:01.046115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:01.046564 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:01.046809 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:01.047057 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 03:36:01.047382 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":148,"threads_started":1}
I20260809 03:36:01.049432 18386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566b0293b66f485383de9917f2f22a7e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43485 } }
I20260809 03:36:01.049798 18386 leader_election.cc:302] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 566b0293b66f485383de9917f2f22a7e; no voters:
I20260809 03:36:01.050166 18386 leader_election.cc:288] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:01.050252 18394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:01.050462 18394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [term 1 LEADER]: Becoming Leader. State: Replica: 566b0293b66f485383de9917f2f22a7e, State: Running, Role: LEADER
I20260809 03:36:01.050781 18394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566b0293b66f485383de9917f2f22a7e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43485 } }
I20260809 03:36:01.051599 18386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:01.052709 18399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 566b0293b66f485383de9917f2f22a7e. Latest consensus state: current_term: 1 leader_uuid: "566b0293b66f485383de9917f2f22a7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566b0293b66f485383de9917f2f22a7e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43485 } } }
I20260809 03:36:01.052760 18395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "566b0293b66f485383de9917f2f22a7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566b0293b66f485383de9917f2f22a7e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43485 } } }
I20260809 03:36:01.052839 18399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:01.052868 18395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:01.054211 18410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:01.057500 18410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:01.063203 18410 catalog_manager.cc:1347] Generated new cluster ID: c41be377ef1a4a4e9878a62348d745b3
I20260809 03:36:01.063283 18410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:01.084698 18410 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:01.085759 18410 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:01.100831 18410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e: Generated new TSK 0
I20260809 03:36:01.101660 18410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:36:01.147714 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43485
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:01.266474 18428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:01.266745 18428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:01.266789 18428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:01.271010 18428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:01.271083 18428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:01.271178 18428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:36:01.276774 18428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:36:01.277899 18428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:01.278988 18428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:01.289276 18433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:01.289269 18434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:01.289288 18438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:01.289685 18428 server_base.cc:1034] running on GCE node
I20260809 03:36:01.290109 18428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:01.290733 18428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:01.291896 18428 hybrid_clock.cc:648] HybridClock initialized: now 1786246561291859 us; error 49 us; skew 500 ppm
Aug 09 03:36:01 dist-test-slave-jq4t krb5kdc[18272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:01.298653 18428 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:36:01.299798 18428 webserver.cc:460] Webserver started at http://127.1.224.65:46833/ using document root <none> and password file <none>
I20260809 03:36:01.300554 18428 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:01.300625 18428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:01.300814 18428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:01.302927 18428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "dac921a0b7204f9984f4013e0bb1a8d3"
format_stamp: "Formatted at 2026-08-09 03:36:01 on dist-test-slave-jq4t"
server_key: "4d7d2b26549ae475806717835dffa92e"
server_key_iv: "7107a2311027db2af77722ef63d2e7b8"
server_key_version: "encryptionkey@0"
I20260809 03:36:01.303508 18428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "dac921a0b7204f9984f4013e0bb1a8d3"
format_stamp: "Formatted at 2026-08-09 03:36:01 on dist-test-slave-jq4t"
server_key: "4d7d2b26549ae475806717835dffa92e"
server_key_iv: "7107a2311027db2af77722ef63d2e7b8"
server_key_version: "encryptionkey@0"
I20260809 03:36:01.307361 18428 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260809 03:36:01.309952 18447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:01.311261 18428 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:36:01.311412 18428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "dac921a0b7204f9984f4013e0bb1a8d3"
format_stamp: "Formatted at 2026-08-09 03:36:01 on dist-test-slave-jq4t"
server_key: "4d7d2b26549ae475806717835dffa92e"
server_key_iv: "7107a2311027db2af77722ef63d2e7b8"
server_key_version: "encryptionkey@0"
I20260809 03:36:01.311527 18428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:01.329777 18428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:01.333449 18428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:01.333680 18428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:01.334355 18428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:01.335240 18428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:01.335290 18428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:01.335331 18428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:01.335346 18428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:01.344565 18428 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:43141
I20260809 03:36:01.344588 18610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:43141 every 8 connection(s)
I20260809 03:36:01.345630 18428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:36:01.346009 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 18428
I20260809 03:36:01.346144 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:36:01.346527 1921 external_mini_cluster.cc:1428] Setting key 6757010c7eb0ce5faa4d3da977d58304
Aug 09 03:36:01 dist-test-slave-jq4t krb5kdc[18272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:36:01.362668 18390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.347589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:57705 (local address 127.1.224.126:43485)
0809 03:36:01.347735 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:01.347738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:01.348908 (+ 1170us) server_negotiation.cc:407] Connection header received
0809 03:36:01.350486 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:01.350489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:01.350537 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:01.350607 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:01.352385 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.353061 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.355066 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.355307 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.355407 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.358602 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:01.358606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:01.358608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:01.358634 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:01.360111 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:01.360803 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:01.360808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:01.360810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:01.360904 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:01.361318 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:01.361321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:01.361323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:01.361595 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:01.361712 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:01.362321 (+ 609us) server_negotiation.cc:299] Negotiation successful
0809 03:36:01.362480 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:36:01.363605 18614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.348163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:43485 (local address 127.1.224.65:57705)
0809 03:36:01.348737 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:01.348794 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:01.350237 (+ 1443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:01.350802 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:01.350813 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:01.351430 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:01.352228 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.352251 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.353225 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.353232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.354890 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.354907 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.355504 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.355512 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.355774 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.356544 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:01.356568 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:01.358376 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:01.360249 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:01.360256 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:01.360274 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:01.360679 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:01.361017 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:01.361020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:01.361022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:01.361219 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:01.361711 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:01.361720 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:01.362048 (+ 328us) client_negotiation.cc:769] Sending connection context
0809 03:36:01.362257 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 03:36:01.362635 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":149,"threads_started":1}
I20260809 03:36:01.365067 18611 heartbeater.cc:347] Connected to a master server at 127.1.224.126:43485
I20260809 03:36:01.365362 18611 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:01.365916 18611 heartbeater.cc:511] Master 127.1.224.126:43485 requested a full tablet report, sending...
I20260809 03:36:01.367815 18338 ts_manager.cc:194] Registered new tserver with Master: dac921a0b7204f9984f4013e0bb1a8d3 (127.1.224.65:43141)
I20260809 03:36:01.369686 18338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:57705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:36:01.380524 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43485
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:01.502058 18624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:01.502365 18624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:01.502410 18624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:01.506776 18624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:01.506852 18624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:01.506938 18624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:36:01.512633 18624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:36:01.513868 18624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:01.515105 18624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:01.526064 18635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:01.526265 18624 server_base.cc:1034] running on GCE node
W20260809 03:36:01.526064 18629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:01.526099 18630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:01.526948 18624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:01.527644 18624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:01.528815 18624 hybrid_clock.cc:648] HybridClock initialized: now 1786246561528792 us; error 37 us; skew 500 ppm
Aug 09 03:36:01 dist-test-slave-jq4t krb5kdc[18272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:01.531963 18624 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:36:01.533066 18624 webserver.cc:460] Webserver started at http://127.1.224.66:39321/ using document root <none> and password file <none>
I20260809 03:36:01.533710 18624 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:01.533778 18624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:01.534062 18624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:01.536314 18624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "0be8f63466b44365ab11da4d1a3c001f"
format_stamp: "Formatted at 2026-08-09 03:36:01 on dist-test-slave-jq4t"
server_key: "b220186f205b18558ce25c58bc98139e"
server_key_iv: "cc5243ff673b765fc668d2e07ec888eb"
server_key_version: "encryptionkey@0"
I20260809 03:36:01.537043 18624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "0be8f63466b44365ab11da4d1a3c001f"
format_stamp: "Formatted at 2026-08-09 03:36:01 on dist-test-slave-jq4t"
server_key: "b220186f205b18558ce25c58bc98139e"
server_key_iv: "cc5243ff673b765fc668d2e07ec888eb"
server_key_version: "encryptionkey@0"
I20260809 03:36:01.541170 18624 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:36:01.543936 18645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:01.545385 18624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:01.545555 18624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "0be8f63466b44365ab11da4d1a3c001f"
format_stamp: "Formatted at 2026-08-09 03:36:01 on dist-test-slave-jq4t"
server_key: "b220186f205b18558ce25c58bc98139e"
server_key_iv: "cc5243ff673b765fc668d2e07ec888eb"
server_key_version: "encryptionkey@0"
I20260809 03:36:01.545686 18624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:01.561647 18624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:01.564778 18624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:01.565007 18624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:01.565644 18624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:01.566543 18624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:01.566619 18624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:01.566696 18624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:01.566740 18624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:01.575817 18624 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:37287
I20260809 03:36:01.575845 18825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:37287 every 8 connection(s)
I20260809 03:36:01.576865 18624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:36:01.580190 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 18624
I20260809 03:36:01.580310 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:36:01.580724 1921 external_mini_cluster.cc:1428] Setting key 980a32450a71327fa6c8767296b239b4
Aug 09 03:36:01 dist-test-slave-jq4t krb5kdc[18272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:36:01.593120 18390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.578831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:48775 (local address 127.1.224.126:43485)
0809 03:36:01.579004 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:01.579008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:01.579864 (+ 856us) server_negotiation.cc:407] Connection header received
0809 03:36:01.581764 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:01.581767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:01.581826 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:01.581940 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:01.583281 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.583820 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.585638 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.585865 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.585952 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.589056 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:01.589059 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:01.589062 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:01.589088 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:01.590620 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:01.591217 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:01.591221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:01.591222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:01.591303 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:01.591746 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:01.591750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:01.591752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:01.591970 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:01.592100 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:01.592786 (+ 686us) server_negotiation.cc:299] Negotiation successful
0809 03:36:01.592929 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 03:36:01.594128 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.579218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:43485 (local address 127.1.224.66:48775)
0809 03:36:01.579699 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:01.579745 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:01.581525 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:01.582154 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:01.582164 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:01.582601 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:01.583114 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.583129 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.583963 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.583967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.585500 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.585517 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.586037 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.586044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.586288 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.586974 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:01.587001 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:01.588850 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:01.590756 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:01.590763 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:01.590781 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:01.591089 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:01.591405 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:01.591408 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:01.591410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:01.591628 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:01.592098 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:01.592106 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:01.592434 (+ 328us) client_negotiation.cc:769] Sending connection context
0809 03:36:01.592724 (+ 290us) client_negotiation.cc:240] Negotiation successful
0809 03:36:01.593105 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
I20260809 03:36:01.595404 18828 heartbeater.cc:347] Connected to a master server at 127.1.224.126:43485
I20260809 03:36:01.595678 18828 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:01.596237 18828 heartbeater.cc:511] Master 127.1.224.126:43485 requested a full tablet report, sending...
I20260809 03:36:01.597638 18338 ts_manager.cc:194] Registered new tserver with Master: 0be8f63466b44365ab11da4d1a3c001f (127.1.224.66:37287)
I20260809 03:36:01.598392 18338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:48775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:36:01.611845 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43485
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:01.735456 18833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:01.735733 18833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:01.735777 18833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:01.739961 18833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:01.740022 18833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:01.740079 18833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:36:01.745500 18833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:36:01.746618 18833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:01.747638 18833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:01.758095 18845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:01.758090 18841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:01.758265 18833 server_base.cc:1034] running on GCE node
W20260809 03:36:01.758416 18839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:01.758845 18833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:01.759507 18833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:01.760694 18833 hybrid_clock.cc:648] HybridClock initialized: now 1786246561760674 us; error 32 us; skew 500 ppm
Aug 09 03:36:01 dist-test-slave-jq4t krb5kdc[18272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:01.763511 18833 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:36:01.764467 18833 webserver.cc:460] Webserver started at http://127.1.224.67:45271/ using document root <none> and password file <none>
I20260809 03:36:01.765126 18833 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:01.765223 18833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:01.765468 18833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:36:01.767527 18833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b"
format_stamp: "Formatted at 2026-08-09 03:36:01 on dist-test-slave-jq4t"
server_key: "7a691f5e009222419003f0198f2ea04f"
server_key_iv: "c89c1e24ce6793098f604ebbf194ee79"
server_key_version: "encryptionkey@0"
I20260809 03:36:01.768124 18833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b"
format_stamp: "Formatted at 2026-08-09 03:36:01 on dist-test-slave-jq4t"
server_key: "7a691f5e009222419003f0198f2ea04f"
server_key_iv: "c89c1e24ce6793098f604ebbf194ee79"
server_key_version: "encryptionkey@0"
I20260809 03:36:01.772071 18833 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 03:36:01.774646 18856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:01.775879 18833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:36:01.776053 18833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b"
format_stamp: "Formatted at 2026-08-09 03:36:01 on dist-test-slave-jq4t"
server_key: "7a691f5e009222419003f0198f2ea04f"
server_key_iv: "c89c1e24ce6793098f604ebbf194ee79"
server_key_version: "encryptionkey@0"
I20260809 03:36:01.776176 18833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:01.795665 18833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:01.798810 18833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:01.799045 18833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:01.799702 18833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:01.800637 18833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:01.800712 18833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:01.800792 18833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:01.800848 18833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:01.810545 18833 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:34497
I20260809 03:36:01.810575 19028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:34497 every 8 connection(s)
I20260809 03:36:01.811529 18833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:36:01.820412 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 18833
I20260809 03:36:01.820628 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:36:01.820964 1921 external_mini_cluster.cc:1428] Setting key 504335742ab8086bba29da33a5048a65
Aug 09 03:36:01 dist-test-slave-jq4t krb5kdc[18272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:36:01.826206 18390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.813449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:52001 (local address 127.1.224.126:43485)
0809 03:36:01.813630 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:01.813635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:01.814534 (+ 899us) server_negotiation.cc:407] Connection header received
0809 03:36:01.816105 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:01.816109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:01.816161 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:01.816239 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:01.817480 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.818016 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.819287 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.819501 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.819578 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.822675 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:01.822679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:01.822681 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:01.822706 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:01.824210 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:01.824786 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:01.824789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:01.824791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:01.824871 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:01.825192 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:01.825195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:01.825196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:01.825392 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:01.825470 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:01.825973 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 03:36:01.826082 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 03:36:01.827062 19034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.813851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:43485 (local address 127.1.224.67:52001)
0809 03:36:01.814362 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:01.814407 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:01.815888 (+ 1481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:01.816389 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:01.816396 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:01.816826 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:01.817343 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.817357 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.818183 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.818186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.819168 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.819179 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.819646 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.819649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.819800 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.820575 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:01.820601 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:01.822509 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:01.824321 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:01.824326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:01.824339 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:01.824696 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:01.824956 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:01.824958 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:01.824960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:01.825104 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:01.825480 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:01.825485 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:01.825741 (+ 256us) client_negotiation.cc:769] Sending connection context
0809 03:36:01.825896 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 03:36:01.826177 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":125,"threads_started":1}
I20260809 03:36:01.828248 19032 heartbeater.cc:347] Connected to a master server at 127.1.224.126:43485
I20260809 03:36:01.828559 19032 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:01.829173 19032 heartbeater.cc:511] Master 127.1.224.126:43485 requested a full tablet report, sending...
I20260809 03:36:01.830283 18338 ts_manager.cc:194] Registered new tserver with Master: 5f4a1856ce554a0da7ee16ea7b8f9b3b (127.1.224.67:34497)
I20260809 03:36:01.830904 18338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:52001
I20260809 03:36:01.835127 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:36:01.844113 18390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.837154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39632 (local address 127.1.224.126:43485)
0809 03:36:01.837295 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:01.837299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:01.837430 (+ 131us) server_negotiation.cc:407] Connection header received
0809 03:36:01.837552 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:01.837554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:01.837592 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:01.837652 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:01.838314 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.838823 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.839660 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.839899 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.839989 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.841395 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:01.841399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:01.841401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:01.841423 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:01.842690 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:01.843068 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:01.843071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:01.843072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:01.843143 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:01.843393 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:01.843395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:01.843397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:01.843605 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:01.843707 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:01.843903 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 03:36:01.843996 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:36:01.847280 18338 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:36:01.850620 18338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:36:01.866230 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.860102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:43141 (local address 127.0.0.1:39254)
0809 03:36:01.860744 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:01.860797 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:01.860961 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:01.861682 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:01.861688 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:01.861709 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:01.861823 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.861827 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.863155 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.863161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.864457 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.864467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.865620 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.865624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.865732 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.865790 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 03:36:01.865902 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 03:36:01.866036 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":15,"spinlock_wait_cycles":5504,"thread_start_us":114,"threads_started":1}
I20260809 03:36:01.866398 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.860407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56364 (local address 127.1.224.66:37287)
0809 03:36:01.861965 (+ 1558us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:01.861976 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:01.861993 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:36:01.862060 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:01.862063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:01.862227 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:01.862382 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:01.862623 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.863677 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.865094 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.865728 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.865888 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.866029 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:01.866146 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 03:36:01.866245 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1471,"thread_start_us":184,"threads_started":1}
I20260809 03:36:01.866720 19047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.859675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:34497 (local address 127.0.0.1:45018)
0809 03:36:01.860995 (+ 1320us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:01.861009 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:01.861098 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:01.861987 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:01.861990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:01.862008 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:01.862100 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.862107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.863610 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.863613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.864859 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.864871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.866275 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.866277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.866392 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.866425 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 03:36:01.866503 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 03:36:01.866564 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":96,"threads_started":1}
I20260809 03:36:01.866724 19051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.859836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:37287 (local address 127.0.0.1:56364)
0809 03:36:01.860745 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:01.860799 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:01.860951 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:01.862374 (+ 1423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:01.862377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:01.862396 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:01.862487 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.862496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.863818 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.863821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.864961 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.864970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.865909 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.865913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.866001 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.866025 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:36:01.866111 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 03:36:01.866190 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":2,"spinlock_wait_cycles":11392,"thread_start_us":120,"threads_started":1}
I20260809 03:36:01.866883 19053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.860574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39254 (local address 127.1.224.65:43141)
0809 03:36:01.861275 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:01.861290 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:01.861311 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:36:01.861379 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:01.861383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:01.861525 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:01.861669 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:01.861959 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.863022 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.864687 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.865470 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.865596 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.865768 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:01.866616 (+ 848us) server_negotiation.cc:299] Negotiation successful
0809 03:36:01.866720 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":126,"threads_started":1}
I20260809 03:36:01.867390 19049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.859837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45018 (local address 127.1.224.67:34497)
0809 03:36:01.860927 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:01.860940 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:01.861194 (+ 254us) server_negotiation.cc:407] Connection header received
0809 03:36:01.861278 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:01.861286 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:01.861496 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:01.861669 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:01.862231 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.863450 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.865000 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.865788 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.865922 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.866077 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:01.867164 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0809 03:36:01.867269 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":983,"thread_start_us":157,"threads_started":1}
I20260809 03:36:01.869923 18524 tablet_service.cc:1467] Processing CreateTablet for tablet 0e22e91323ea4a8aa307d51cf2991d6f (DEFAULT_TABLE table=test-table [id=26b6ede690874b9a907da6941db96987]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:01.870194 18929 tablet_service.cc:1467] Processing CreateTablet for tablet 0e22e91323ea4a8aa307d51cf2991d6f (DEFAULT_TABLE table=test-table [id=26b6ede690874b9a907da6941db96987]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:01.870194 18725 tablet_service.cc:1467] Processing CreateTablet for tablet 0e22e91323ea4a8aa307d51cf2991d6f (DEFAULT_TABLE table=test-table [id=26b6ede690874b9a907da6941db96987]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:01.871076 18524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e22e91323ea4a8aa307d51cf2991d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:01.871256 18929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e22e91323ea4a8aa307d51cf2991d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:01.871250 18725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e22e91323ea4a8aa307d51cf2991d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:01.877831 19054 tablet_bootstrap.cc:492] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3: Bootstrap starting.
I20260809 03:36:01.879400 19055 tablet_bootstrap.cc:492] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f: Bootstrap starting.
I20260809 03:36:01.880333 19054 tablet_bootstrap.cc:654] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:01.880834 19056 tablet_bootstrap.cc:492] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b: Bootstrap starting.
I20260809 03:36:01.881361 19054 log.cc:824] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:01.881935 19055 tablet_bootstrap.cc:654] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:01.882845 19055 log.cc:824] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:01.883514 19056 tablet_bootstrap.cc:654] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:01.884195 19054 tablet_bootstrap.cc:492] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3: No bootstrap required, opened a new log
I20260809 03:36:01.884346 19056 log.cc:824] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:01.884429 19054 ts_tablet_manager.cc:1397] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 03:36:01.884673 19055 tablet_bootstrap.cc:492] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f: No bootstrap required, opened a new log
I20260809 03:36:01.884871 19055 ts_tablet_manager.cc:1397] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:36:01.886412 19056 tablet_bootstrap.cc:492] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b: No bootstrap required, opened a new log
I20260809 03:36:01.886615 19056 ts_tablet_manager.cc:1397] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:36:01.888031 19054 raft_consensus.cc:348] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 34497 } } peers { permanent_uuid: "0be8f63466b44365ab11da4d1a3c001f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37287 } } peers { permanent_uuid: "dac921a0b7204f9984f4013e0bb1a8d3" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43141 } }
I20260809 03:36:01.888262 19054 raft_consensus.cc:374] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:01.888309 19054 raft_consensus.cc:729] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dac921a0b7204f9984f4013e0bb1a8d3, State: Initialized, Role: FOLLOWER
I20260809 03:36:01.888818 19054 consensus_queue.cc:260] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 34497 } } peers { permanent_uuid: "0be8f63466b44365ab11da4d1a3c001f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37287 } } peers { permanent_uuid: "dac921a0b7204f9984f4013e0bb1a8d3" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43141 } }
I20260809 03:36:01.888770 19055 raft_consensus.cc:348] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 34497 } } peers { permanent_uuid: "0be8f63466b44365ab11da4d1a3c001f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37287 } } peers { permanent_uuid: "dac921a0b7204f9984f4013e0bb1a8d3" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43141 } }
I20260809 03:36:01.889025 19055 raft_consensus.cc:374] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:01.889102 19055 raft_consensus.cc:729] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be8f63466b44365ab11da4d1a3c001f, State: Initialized, Role: FOLLOWER
I20260809 03:36:01.889528 19055 consensus_queue.cc:260] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 34497 } } peers { permanent_uuid: "0be8f63466b44365ab11da4d1a3c001f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37287 } } peers { permanent_uuid: "dac921a0b7204f9984f4013e0bb1a8d3" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43141 } }
I20260809 03:36:01.890447 19054 ts_tablet_manager.cc:1428] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260809 03:36:01.890507 19055 ts_tablet_manager.cc:1428] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:36:01.890589 19056 raft_consensus.cc:348] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 34497 } } peers { permanent_uuid: "0be8f63466b44365ab11da4d1a3c001f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37287 } } peers { permanent_uuid: "dac921a0b7204f9984f4013e0bb1a8d3" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43141 } }
I20260809 03:36:01.890853 19056 raft_consensus.cc:374] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:01.890933 19056 raft_consensus.cc:729] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f4a1856ce554a0da7ee16ea7b8f9b3b, State: Initialized, Role: FOLLOWER
I20260809 03:36:01.890930 18828 heartbeater.cc:503] Master 127.1.224.126:43485 was elected leader, sending a full tablet report...
I20260809 03:36:01.891310 19056 consensus_queue.cc:260] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 34497 } } peers { permanent_uuid: "0be8f63466b44365ab11da4d1a3c001f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37287 } } peers { permanent_uuid: "dac921a0b7204f9984f4013e0bb1a8d3" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43141 } }
I20260809 03:36:01.891999 19032 heartbeater.cc:503] Master 127.1.224.126:43485 was elected leader, sending a full tablet report...
I20260809 03:36:01.892170 18611 heartbeater.cc:503] Master 127.1.224.126:43485 was elected leader, sending a full tablet report...
I20260809 03:36:01.892241 19056 ts_tablet_manager.cc:1428] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:36:01.968974 19063 raft_consensus.cc:482] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:01.969283 19063 raft_consensus.cc:504] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 34497 } } peers { permanent_uuid: "0be8f63466b44365ab11da4d1a3c001f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37287 } } peers { permanent_uuid: "dac921a0b7204f9984f4013e0bb1a8d3" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43141 } }
I20260809 03:36:01.970644 19063 leader_election.cc:288] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0be8f63466b44365ab11da4d1a3c001f (127.1.224.66:37287), dac921a0b7204f9984f4013e0bb1a8d3 (127.1.224.65:43141)
I20260809 03:36:01.975911 19053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.971291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:55175 (local address 127.1.224.65:43141)
0809 03:36:01.971439 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:01.971446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:01.971601 (+ 155us) server_negotiation.cc:407] Connection header received
0809 03:36:01.971692 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:01.971695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:01.971735 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:01.971796 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:01.972086 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.972824 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.974520 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.975172 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.975292 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.975351 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:01.975694 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 03:36:01.975770 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:36:01.975911 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.971062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:54449 (local address 127.1.224.66:37287)
0809 03:36:01.971241 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:01.971246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:01.971265 (+ 19us) server_negotiation.cc:407] Connection header received
0809 03:36:01.971329 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:01.971336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:01.971412 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:01.971516 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:01.971872 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.972708 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.974522 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.975152 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.975293 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.975351 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:01.975694 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 03:36:01.975770 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 03:36:01.975937 19034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.970912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:37287 (local address 127.1.224.67:54449)
0809 03:36:01.971064 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:01.971085 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:01.971196 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:01.971557 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:01.971563 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:01.971588 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:01.971735 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.971740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.972817 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.972820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.974353 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.974366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.975296 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.975300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.975499 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.975586 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 03:36:01.975661 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 03:36:01.975769 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":262}
I20260809 03:36:01.975946 19065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:01.971043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:43141 (local address 127.1.224.67:55175)
0809 03:36:01.971489 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:01.971506 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:01.971604 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:01.971835 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:01.971840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:01.971864 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:01.971967 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.971981 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.972930 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.972933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.974353 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:01.974366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:01.975296 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:01.975300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:01.975519 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:01.975579 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 03:36:01.975661 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 03:36:01.975769 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":133,"spinlock_wait_cycles":3328,"thread_start_us":146,"threads_started":1}
I20260809 03:36:01.976802 18752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e22e91323ea4a8aa307d51cf2991d6f" candidate_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be8f63466b44365ab11da4d1a3c001f" is_pre_election: true
I20260809 03:36:01.976915 18551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e22e91323ea4a8aa307d51cf2991d6f" candidate_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dac921a0b7204f9984f4013e0bb1a8d3" is_pre_election: true
I20260809 03:36:01.977185 18752 raft_consensus.cc:2454] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f4a1856ce554a0da7ee16ea7b8f9b3b in term 0.
I20260809 03:36:01.977226 18551 raft_consensus.cc:2454] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f4a1856ce554a0da7ee16ea7b8f9b3b in term 0.
I20260809 03:36:01.977731 18858 leader_election.cc:302] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0be8f63466b44365ab11da4d1a3c001f, 5f4a1856ce554a0da7ee16ea7b8f9b3b, dac921a0b7204f9984f4013e0bb1a8d3; no voters:
I20260809 03:36:01.978021 19063 raft_consensus.cc:2781] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:36:01.978125 19063 raft_consensus.cc:482] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:36:01.978188 19063 raft_consensus.cc:3037] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:01.979308 19063 raft_consensus.cc:504] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 34497 } } peers { permanent_uuid: "0be8f63466b44365ab11da4d1a3c001f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37287 } } peers { permanent_uuid: "dac921a0b7204f9984f4013e0bb1a8d3" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43141 } }
I20260809 03:36:01.979732 19063 leader_election.cc:288] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [CANDIDATE]: Term 1 election: Requested vote from peers 0be8f63466b44365ab11da4d1a3c001f (127.1.224.66:37287), dac921a0b7204f9984f4013e0bb1a8d3 (127.1.224.65:43141)
I20260809 03:36:01.980088 18752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e22e91323ea4a8aa307d51cf2991d6f" candidate_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be8f63466b44365ab11da4d1a3c001f"
I20260809 03:36:01.980129 18551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e22e91323ea4a8aa307d51cf2991d6f" candidate_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dac921a0b7204f9984f4013e0bb1a8d3"
I20260809 03:36:01.980230 18752 raft_consensus.cc:3037] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:01.980259 18551 raft_consensus.cc:3037] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:01.981482 18752 raft_consensus.cc:2454] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f4a1856ce554a0da7ee16ea7b8f9b3b in term 1.
I20260809 03:36:01.981503 18551 raft_consensus.cc:2454] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f4a1856ce554a0da7ee16ea7b8f9b3b in term 1.
I20260809 03:36:01.981789 18860 leader_election.cc:302] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0be8f63466b44365ab11da4d1a3c001f, 5f4a1856ce554a0da7ee16ea7b8f9b3b; no voters:
I20260809 03:36:01.982110 19063 raft_consensus.cc:2781] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:01.982388 19063 raft_consensus.cc:686] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 1 LEADER]: Becoming Leader. State: Replica: 5f4a1856ce554a0da7ee16ea7b8f9b3b, State: Running, Role: LEADER
I20260809 03:36:01.982775 19063 consensus_queue.cc:237] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 34497 } } peers { permanent_uuid: "0be8f63466b44365ab11da4d1a3c001f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37287 } } peers { permanent_uuid: "dac921a0b7204f9984f4013e0bb1a8d3" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43141 } }
I20260809 03:36:01.985953 18338 catalog_manager.cc:5495] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f4a1856ce554a0da7ee16ea7b8f9b3b (127.1.224.67). New cstate: current_term: 1 leader_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a1856ce554a0da7ee16ea7b8f9b3b" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 34497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0be8f63466b44365ab11da4d1a3c001f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dac921a0b7204f9984f4013e0bb1a8d3" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43141 } health_report { overall_health: UNKNOWN } } }
I20260809 03:36:02.021347 19049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:02.017480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.1.224.67:34497)
0809 03:36:02.017654 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:02.017658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:02.017674 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:36:02.017817 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:02.017820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:02.017900 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:02.018026 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:36:02.018293 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:02.018846 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:02.019753 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:02.019986 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:02.020102 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:02.020430 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:36:02.020916 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:36:02.021058 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:02.021106 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 03:36:02.021169 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 03:36:02.027838 18752 raft_consensus.cc:1261] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f [term 1 FOLLOWER]: Refusing update from remote peer 5f4a1856ce554a0da7ee16ea7b8f9b3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:02.027876 18551 raft_consensus.cc:1261] T 0e22e91323ea4a8aa307d51cf2991d6f P dac921a0b7204f9984f4013e0bb1a8d3 [term 1 FOLLOWER]: Refusing update from remote peer 5f4a1856ce554a0da7ee16ea7b8f9b3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:02.028652 19063 consensus_queue.cc:1035] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0be8f63466b44365ab11da4d1a3c001f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:02.028906 19067 consensus_queue.cc:1035] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dac921a0b7204f9984f4013e0bb1a8d3" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:02.038697 19076 mvcc.cc:205] Tried to move back new op lower bound from 7316465918057545728 to 7316465917884096512. Current Snapshot: MvccSnapshot[applied={T|T < 7316465918057545728 or (T in {7316465918057545728})}]
I20260809 03:36:02.048746 18337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:02.048967 18337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:02.051674 18337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 566b0293b66f485383de9917f2f22a7e: Sending DeleteTablet for 3 replicas of tablet 0e22e91323ea4a8aa307d51cf2991d6f
I20260809 03:36:02.052561 18929 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e22e91323ea4a8aa307d51cf2991d6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:02 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:45018
I20260809 03:36:02.052565 18725 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e22e91323ea4a8aa307d51cf2991d6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:02 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:56364
I20260809 03:36:02.052845 18524 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e22e91323ea4a8aa307d51cf2991d6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:02 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:39254
I20260809 03:36:02.053133 19088 tablet_replica.cc:330] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f: stopping tablet replica
I20260809 03:36:02.053514 19088 raft_consensus.cc:2229] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:36:02.053884 19088 raft_consensus.cc:2258] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:02.054222 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 18428
I20260809 03:36:02.059319 19088 ts_tablet_manager.cc:1905] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:02.062198 19088 ts_tablet_manager.cc:1918] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:02.062338 19088 log.cc:1196] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/0e22e91323ea4a8aa307d51cf2991d6f
I20260809 03:36:02.062637 19088 ts_tablet_manager.cc:1939] T 0e22e91323ea4a8aa307d51cf2991d6f P 0be8f63466b44365ab11da4d1a3c001f: Deleting consensus metadata
I20260809 03:36:02.063912 19090 tablet_replica.cc:330] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b: stopping tablet replica
I20260809 03:36:02.063933 18316 catalog_manager.cc:4841] TS 0be8f63466b44365ab11da4d1a3c001f (127.1.224.66:37287): tablet 0e22e91323ea4a8aa307d51cf2991d6f (table test-table [id=26b6ede690874b9a907da6941db96987]) successfully deleted
I20260809 03:36:02.064266 19090 raft_consensus.cc:2229] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:36:02.064785 19090 raft_consensus.cc:2258] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:02.066069 19090 ts_tablet_manager.cc:1905] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:02.068214 19090 ts_tablet_manager.cc:1918] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:02.068336 19090 log.cc:1196] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/0e22e91323ea4a8aa307d51cf2991d6f
I20260809 03:36:02.068737 19090 ts_tablet_manager.cc:1939] T 0e22e91323ea4a8aa307d51cf2991d6f P 5f4a1856ce554a0da7ee16ea7b8f9b3b: Deleting consensus metadata
I20260809 03:36:02.069950 18317 catalog_manager.cc:4841] TS 5f4a1856ce554a0da7ee16ea7b8f9b3b (127.1.224.67:34497): tablet 0e22e91323ea4a8aa307d51cf2991d6f (table test-table [id=26b6ede690874b9a907da6941db96987]) successfully deleted
W20260809 03:36:02.072849 18314 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:43141 (error 108)
I20260809 03:36:02.073846 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 18624
I20260809 03:36:02.073975 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:02.073666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:43141 (local address 127.0.0.1:39264)
0809 03:36:02.073815 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:02.073885 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:43141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 03:36:02.082477 18314 catalog_manager.cc:4584] TS dac921a0b7204f9984f4013e0bb1a8d3 (127.1.224.65:43141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e22e91323ea4a8aa307d51cf2991d6f: Network error: Client connection negotiation failed: client connection to 127.1.224.65:43141: connect: Connection refused (error 111)
I20260809 03:36:02.106765 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 18833
I20260809 03:36:02.120630 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 18290
2026-08-09T03:36:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2671 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:02 dist-test-slave-jq4t krb5kdc[19097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:02 dist-test-slave-jq4t krb5kdc[19097](info): set up 2 sockets
Aug 09 03:36:02 dist-test-slave-jq4t krb5kdc[19097](info): commencing operation
krb5kdc: starting...
W20260809 03:36:03.392396 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:36:03 dist-test-slave-jq4t krb5kdc[19097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:03Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/rangeradmin_127.1.224.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/rangeradmin_127.1.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/rangerlookup_127.1.224.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/rangerlookup_127.1.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/HTTP_127.1.224.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/HTTP_127.1.224.84@KRBTEST.COM.keytab.
I20260809 03:36:03.575902 1921 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfj0rOk/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/postgres -l logfile start
2026-08-09 03:36:06.195 UTC [19139] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 03:36:06.196 UTC [19139] LOG: listening on IPv4 address "127.1.224.84", port 60397
2026-08-09 03:36:06.199 UTC [19139] LOG: listening on Unix socket "/tmp/.s.PGSQL.60397"
2026-08-09 03:36:06.222 UTC [19144] LOG: database system was shut down at 2026-08-09 03:36:04 UTC
2026-08-09 03:36:06.225 UTC [19139] LOG: database system is ready to accept connections
I20260809 03:36:07.400316 1921 mini_postgres.cc:96] Postgres bound to 60397
2026-08-09 03:36:07.410 UTC [19152] FATAL: database "slave" does not exist
127.1.224.84:60397 - accepting connections
I20260809 03:36:07.411062 1921 mini_ranger.cc:161] Starting Ranger...
I20260809 03:36:07.429473 1921 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 03:36:07.894595 1921 mini_ranger.cc:87] Created ranger Postgres database
I20260809 03:36:07.894688 1921 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-admin
2026-08-09 03:36:08,224 [I] DB FLAVOR :POSTGRES
2026-08-09 03:36:08,224 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 03:36:08,225 [I] Checking connection..
2026-08-09 03:36:08,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 03:36:08,491 [I] Checking connection passed.
2026-08-09 03:36:08,491 [I] --------- Verifying version history table ---------
2026-08-09 03:36:08,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:36:08,763 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 03:36:08,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:36:09,004 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 03:36:09,005 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 03:36:09,005 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 03:36:09.269 UTC [19253] WARNING: there is no transaction in progress
2026-08-09 03:36:09,278 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 03:36:09,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:36:09,532 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 03:36:09,533 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 03:36:09,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 03:36:09,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 03:36:10,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jq4t.c.gcp-upstream.internal','N') ;"
2026-08-09 03:36:10,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 03:36:10,544 [I] Table x_portal_user does not exist in database ranger
2026-08-09 03:36:10,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 03:36:10,788 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 03:36:10,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 03:36:11,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 03:36:11,315 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 03:36:11,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 03:36:12.290 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.306 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.322 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.337 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.353 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.370 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.441 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.449 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.463 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.473 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.483 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.498 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.508 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.514 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.521 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:12.528 UTC [19410] WARNING: there is no transaction in progress
2026-08-09 03:36:13,163 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 03:36:13,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 03:36:13,431 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 03:36:13,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 03:36:13,703 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 03:36:13,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 03:36:13,947 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 03:36:13,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 03:36:14,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 03:36:14,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jq4t.c.gcp-upstream.internal';"
2026-08-09 03:36:14,754 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 03:36:14,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 03:36:15,020 [I] DB_PATCHES have already been applied
I20260809 03:36:15.025830 1921 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfj0rOk/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 03:36:15.025900 1921 mini_ranger.cc:193] Using host: 127.1.224.84
I20260809 03:36:15.030686 1921 mini_ranger.cc:227] Ranger admin URL: http://127.1.224.84:60503
Aug 09, 2026 3:36:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:36:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:36:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 3:36:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 3:36:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 3:36:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 3:36:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 03:36:16.378496 1921 mini_ranger.cc:160] Time spent starting Ranger: real 8.967s user 0.003s sys 0.010s
Aug 09, 2026 3:36:37 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 3:36:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 3:36:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 3:36:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 3:36:37 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 3:36:38 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 03:36:39.214126 1921 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/rangerkms_127.1.224.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/rangerkms_127.1.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.224.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.224.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.224.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/HTTP_127.1.224.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.224.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/HTTP_127.1.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 03:36:39 dist-test-slave-jq4t krb5kdc[19097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246599, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 03:36:39.310974 1921 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 03:36:39.322602 1921 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 03:36:40.182547 1921 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 03:36:40.182652 1921 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-kms
I20260809 03:36:40.182773 1921 mini_ranger_kms.cc:227] Using postgres at 127.1.224.84:60397
2026-08-09 03:36:40,688 [I] DB FLAVOR :POSTGRES
2026-08-09 03:36:40,688 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 03:36:40,688 [I] Checking connection
2026-08-09 03:36:40,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 03:36:40,972 [I] connection success
2026-08-09 03:36:40,972 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 03:36:40,972 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 03:36:40,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 03:36:41,234 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 03:36:41,234 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 03:36:41,235 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 03:36:41,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:60397/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 03:36:41,534 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 03:36:42.074877 1921 mini_ranger_kms.cc:310] Created kms service
I20260809 03:36:42.292152 1921 mini_ranger_kms.cc:326] Created kudu user
I20260809 03:36:42.341118 1921 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 03:36:42 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246599, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:36:42.794481 1921 mini_ranger_kms.cc:384] Added ranger policy
I20260809 03:36:42.794687 1921 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 03:36:42.794713 1921 mini_ranger_kms.cc:240] Using host: 127.1.224.84
I20260809 03:36:42.800205 1921 mini_ranger_kms.cc:276] Ranger KMS PID: 19675
I20260809 03:36:42.800380 1921 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.224.84:33205
Aug 09, 2026 3:36:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:36:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:36:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 3:36:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 3:36:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 3:36:44 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 03:36:45 dist-test-slave-jq4t krb5kdc[19097](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246605, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.224.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:45.555516 1921 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.245s user 0.024s sys 0.343s
I20260809 03:36:45.555666 1921 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 03:36:45.555788 1921 mini_ranger_kms.cc:401] 127.1.224.84:33205/kms/v1/keys
Aug 09 03:36:45 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246605, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.224.84@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
Aug 09 03:36:46 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246605, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.224.84@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
Aug 09 03:36:48 dist-test-slave-jq4t krb5kdc[19097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:36:48.608023 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:41595
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:41595
--ranger_config_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.224.84:33205/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:48.766551 19746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:48.766929 19746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:48.767014 19746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:48.771313 19746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:36:48.771440 19746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:48.771474 19746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:48.771517 19746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:36:48.771576 19746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:36:48.777592 19746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.224.84:33205/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:41595
--ranger_config_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:41595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:36:48.778880 19746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:48.780033 19746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:48.791183 19755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:48.791217 19754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:48.791217 19746 server_base.cc:1034] running on GCE node
W20260809 03:36:48.791355 19757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:48.792236 19746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:48.793802 19746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:48.795035 19746 hybrid_clock.cc:648] HybridClock initialized: now 1786246608795014 us; error 54 us; skew 500 ppm
Aug 09 03:36:48 dist-test-slave-jq4t krb5kdc[19097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246608, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:48.802193 19746 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:36:48.803510 19746 webserver.cc:460] Webserver started at http://127.1.224.126:33281/ using document root <none> and password file <none>
I20260809 03:36:48.804168 19746 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:48.804229 19746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:48.804529 19746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:36:48 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246608, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:36:48.856405 19746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "bfc88dc4345d42788e8d2a5fcb173230"
format_stamp: "Formatted at 2026-08-09 03:36:48 on dist-test-slave-jq4t"
server_key: "9e672aa2d9bbf28bee36b3f44ce2e7d0"
server_key_iv: "2c4795b8053c55b4dc49adbc88085ad7"
server_key_version: "kuduclusterkey@0"
I20260809 03:36:48.857344 19746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "bfc88dc4345d42788e8d2a5fcb173230"
format_stamp: "Formatted at 2026-08-09 03:36:48 on dist-test-slave-jq4t"
server_key: "9e672aa2d9bbf28bee36b3f44ce2e7d0"
server_key_iv: "2c4795b8053c55b4dc49adbc88085ad7"
server_key_version: "kuduclusterkey@0"
I20260809 03:36:48.862830 19746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 03:36:48.881282 19768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:48.883101 19746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:36:48.883271 19746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "bfc88dc4345d42788e8d2a5fcb173230"
format_stamp: "Formatted at 2026-08-09 03:36:48 on dist-test-slave-jq4t"
server_key: "9e672aa2d9bbf28bee36b3f44ce2e7d0"
server_key_iv: "2c4795b8053c55b4dc49adbc88085ad7"
server_key_version: "kuduclusterkey@0"
I20260809 03:36:48.883426 19746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:48.899302 19746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:48.903143 19746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:48.903389 19746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:48.914840 19746 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:41595
I20260809 03:36:48.914836 19849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:41595 every 8 connection(s)
I20260809 03:36:48.916329 19746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:36:48.919045 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 19746
I20260809 03:36:48.919195 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:36:48.919545 1921 external_mini_cluster.cc:1428] Setting key b44d0088f391d8a1c41c99de66c8cdfa
I20260809 03:36:48.920243 19850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:48.928355 19850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230: Bootstrap starting.
Aug 09 03:36:48 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:36:48.932165 19850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:48.933247 19850 log.cc:824] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:48.935837 19850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230: No bootstrap required, opened a new log
I20260809 03:36:48.938362 19854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:48.921229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59318 (local address 127.1.224.126:41595)
0809 03:36:48.921773 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:48.921795 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:48.921844 (+ 49us) server_negotiation.cc:407] Connection header received
0809 03:36:48.922816 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:48.922842 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:48.923307 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:48.923752 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:48.924814 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:48.926533 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:48.927836 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:48.928679 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:48.928849 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:48.932243 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:48.932251 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:48.932268 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:48.932332 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:48.935151 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:48.935686 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:48.935692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:48.935700 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:48.935822 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:48.936160 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:48.936164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:48.936165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:48.936707 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:48.936928 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:48.937271 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 03:36:48.937595 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":189,"threads_started":1}
I20260809 03:36:48.939510 19850 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc88dc4345d42788e8d2a5fcb173230" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41595 } }
I20260809 03:36:48.939805 19850 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:48.939869 19850 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc88dc4345d42788e8d2a5fcb173230, State: Initialized, Role: FOLLOWER
I20260809 03:36:48.940357 19850 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc88dc4345d42788e8d2a5fcb173230" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41595 } }
I20260809 03:36:48.940546 19850 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:36:48.940630 19850 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:36:48.940752 19850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:48.941717 19850 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc88dc4345d42788e8d2a5fcb173230" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41595 } }
I20260809 03:36:48.942086 19850 leader_election.cc:302] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfc88dc4345d42788e8d2a5fcb173230; no voters:
I20260809 03:36:48.942425 19850 leader_election.cc:288] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:36:48.942541 19856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:48.942837 19856 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [term 1 LEADER]: Becoming Leader. State: Replica: bfc88dc4345d42788e8d2a5fcb173230, State: Running, Role: LEADER
I20260809 03:36:48.943249 19856 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc88dc4345d42788e8d2a5fcb173230" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41595 } }
I20260809 03:36:48.943794 19850 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:36:48.945309 19858 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfc88dc4345d42788e8d2a5fcb173230. Latest consensus state: current_term: 1 leader_uuid: "bfc88dc4345d42788e8d2a5fcb173230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc88dc4345d42788e8d2a5fcb173230" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41595 } } }
I20260809 03:36:48.945693 19858 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:48.945279 19857 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfc88dc4345d42788e8d2a5fcb173230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc88dc4345d42788e8d2a5fcb173230" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 41595 } } }
I20260809 03:36:48.945937 19857 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230 [sys.catalog]: This master's current role is: LEADER
I20260809 03:36:48.946694 19850 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 03:36:48.947388 19866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:36:48.953920 19866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:36:48.963507 19866 catalog_manager.cc:1347] Generated new cluster ID: 6c58c84a3e4546c8b51358bda688f2b8
I20260809 03:36:48.963595 19866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:36:48.988663 19866 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:36:48.990142 19866 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:36:48.997953 19866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230: Generated new TSK 0
I20260809 03:36:48.999002 19866 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:36:49.748657 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.224.84:33205/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:41595
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:36:49.887358 19926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:49.887795 19926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:49.887873 19926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:49.892989 19926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:49.893087 19926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:49.893181 19926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:36:49.898954 19926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.224.84:33205/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:41595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:36:49.900209 19926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:49.901437 19926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:49.913056 19931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:49.913374 19939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:49.913383 19926 server_base.cc:1034] running on GCE node
W20260809 03:36:49.913153 19934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:49.914211 19926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:49.915089 19926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:49.918401 19926 hybrid_clock.cc:648] HybridClock initialized: now 1786246609918365 us; error 59 us; skew 500 ppm
Aug 09 03:36:49 dist-test-slave-jq4t krb5kdc[19097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246609, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:49.922478 19926 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:36:49.923640 19926 webserver.cc:460] Webserver started at http://127.1.224.65:40657/ using document root <none> and password file <none>
I20260809 03:36:49.924373 19926 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:49.924469 19926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:49.924744 19926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:36:49 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246609, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:36:49.943416 19926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "35cb4370d12f4ba5a0216f4d366d0697"
format_stamp: "Formatted at 2026-08-09 03:36:49 on dist-test-slave-jq4t"
server_key: "1e4e59046252f27fc6d088bfc9925758"
server_key_iv: "f5b280c325861cca4a8199258e59ded7"
server_key_version: "kuduclusterkey@0"
I20260809 03:36:49.944689 19926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "35cb4370d12f4ba5a0216f4d366d0697"
format_stamp: "Formatted at 2026-08-09 03:36:49 on dist-test-slave-jq4t"
server_key: "1e4e59046252f27fc6d088bfc9925758"
server_key_iv: "f5b280c325861cca4a8199258e59ded7"
server_key_version: "kuduclusterkey@0"
I20260809 03:36:49.953369 19926 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.001s
I20260809 03:36:49.974308 19953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:49.976398 19926 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 03:36:49.976595 19926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "35cb4370d12f4ba5a0216f4d366d0697"
format_stamp: "Formatted at 2026-08-09 03:36:49 on dist-test-slave-jq4t"
server_key: "1e4e59046252f27fc6d088bfc9925758"
server_key_iv: "f5b280c325861cca4a8199258e59ded7"
server_key_version: "kuduclusterkey@0"
I20260809 03:36:49.976778 19926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:50.000361 19926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:50.004740 19926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:50.004992 19926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:50.005841 19926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:50.007017 19926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:50.007089 19926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:50.007146 19926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:50.007176 19926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:50.022009 19926 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:33033
I20260809 03:36:50.023326 19926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:36:50.025465 20119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:33033 every 8 connection(s)
I20260809 03:36:50.030143 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 19926
I20260809 03:36:50.030284 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:36:50.030596 1921 external_mini_cluster.cc:1428] Setting key 3464732e4878d855ecfaa295e3b87d72
Aug 09 03:36:50 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246609, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:36:50.046446 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.029261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:56089 (local address 127.1.224.126:41595)
0809 03:36:50.029706 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:50.029712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:50.031510 (+ 1798us) server_negotiation.cc:407] Connection header received
0809 03:36:50.031578 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:50.031580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:50.031639 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:50.031737 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:50.033572 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.034384 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.036541 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.036969 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.037079 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.040854 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:50.040860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:50.040866 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:50.040909 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:50.043104 (+ 2195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:50.043976 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:50.043980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:50.043982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:50.044102 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:50.044752 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:50.044756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:50.044758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:50.045027 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:50.045137 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:50.046078 (+ 941us) server_negotiation.cc:299] Negotiation successful
0809 03:36:50.046232 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":110,"threads_started":1}
I20260809 03:36:50.048040 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.027955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:41595 (local address 127.1.224.65:56089)
0809 03:36:50.029932 (+ 1977us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:50.029995 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:50.031154 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:50.032093 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:50.032105 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:50.032566 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:50.033347 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:50.033372 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.034728 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.034733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:50.036321 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:50.036337 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.037364 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.037369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:50.037650 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.038471 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:50.038501 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:50.040599 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:50.043273 (+ 2674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:50.043285 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:50.043387 (+ 102us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:50.043815 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:50.044229 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:50.044235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:50.044239 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:50.044619 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:50.045163 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:50.045173 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:50.045718 (+ 545us) client_negotiation.cc:769] Sending connection context
0809 03:36:50.045972 (+ 254us) client_negotiation.cc:240] Negotiation successful
0809 03:36:50.046836 (+ 864us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1727,"thread_start_us":146,"threads_started":1}
I20260809 03:36:50.050189 20120 heartbeater.cc:347] Connected to a master server at 127.1.224.126:41595
I20260809 03:36:50.050498 20120 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:50.051268 20120 heartbeater.cc:511] Master 127.1.224.126:41595 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:36:50.053503 19802 ts_manager.cc:194] Registered new tserver with Master: 35cb4370d12f4ba5a0216f4d366d0697 (127.1.224.65:33033)
I20260809 03:36:50.055218 19802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:56089
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:36:50.076406 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.224.84:33205/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:41595
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:36:50 dist-test-slave-jq4t krb5kdc[19097](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 03:36:50.261023 20132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:50.261456 20132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:50.261528 20132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:50.267807 20132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:50.267917 20132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:50.268008 20132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:36:50.276540 20132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.224.84:33205/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:41595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:36:50.278215 20132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:50.279733 20132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:50.293810 20143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:50.294001 20132 server_base.cc:1034] running on GCE node
W20260809 03:36:50.293828 20147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:50.294243 20144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:50.295033 20132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:50.296068 20132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:50.297325 20132 hybrid_clock.cc:648] HybridClock initialized: now 1786246610297293 us; error 57 us; skew 500 ppm
Aug 09 03:36:50 dist-test-slave-jq4t krb5kdc[19097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:50.301548 20132 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:36:50.303004 20132 webserver.cc:460] Webserver started at http://127.1.224.66:34635/ using document root <none> and password file <none>
I20260809 03:36:50.303948 20132 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:50.304050 20132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:50.304328 20132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:36:50 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:36:50.329643 20132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "bdc03b35ad7543f89e4e923d75538c00"
format_stamp: "Formatted at 2026-08-09 03:36:50 on dist-test-slave-jq4t"
server_key: "cfd0cab1108840863de4de84c1edc21b"
server_key_iv: "91e054a4dabd7a7595d1c3317db0801c"
server_key_version: "kuduclusterkey@0"
I20260809 03:36:50.330617 20132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "bdc03b35ad7543f89e4e923d75538c00"
format_stamp: "Formatted at 2026-08-09 03:36:50 on dist-test-slave-jq4t"
server_key: "cfd0cab1108840863de4de84c1edc21b"
server_key_iv: "91e054a4dabd7a7595d1c3317db0801c"
server_key_version: "kuduclusterkey@0"
I20260809 03:36:50.336901 20132 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260809 03:36:50.361541 20157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:50.363188 20132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:36:50.363364 20132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "bdc03b35ad7543f89e4e923d75538c00"
format_stamp: "Formatted at 2026-08-09 03:36:50 on dist-test-slave-jq4t"
server_key: "cfd0cab1108840863de4de84c1edc21b"
server_key_iv: "91e054a4dabd7a7595d1c3317db0801c"
server_key_version: "kuduclusterkey@0"
I20260809 03:36:50.363516 20132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:50.401660 20132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:50.406389 20132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:50.406652 20132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:50.407765 20132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:50.409273 20132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:50.409364 20132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:50.409433 20132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:50.409471 20132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:50.424532 20132 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:42003
I20260809 03:36:50.425879 20132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:36:50.426856 20320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:42003 every 8 connection(s)
I20260809 03:36:50.429875 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 20132
I20260809 03:36:50.430017 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:36:50.430352 1921 external_mini_cluster.cc:1428] Setting key e5fae09b3aa26aac17cef4aeebc7e831
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
Aug 09 03:36:50 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
I20260809 03:36:50.453866 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.432534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:57491 (local address 127.1.224.126:41595)
0809 03:36:50.432680 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:50.432685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:50.435426 (+ 2741us) server_negotiation.cc:407] Connection header received
0809 03:36:50.436909 (+ 1483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:50.436913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:50.436974 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:50.437074 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:50.438881 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.439747 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.441672 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.442019 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.442108 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.448937 (+ 6829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:50.448944 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:50.448950 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:50.448991 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:50.450981 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:50.451725 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:50.451729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:50.451731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:50.451825 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:50.452275 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:50.452278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:50.452280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:50.452560 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:50.452657 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:50.453497 (+ 840us) server_negotiation.cc:299] Negotiation successful
0809 03:36:50.453665 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:36:50.455986 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.431168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:41595 (local address 127.1.224.66:57491)
0809 03:36:50.435226 (+ 4058us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:50.435297 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:50.436631 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:50.437448 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:50.437459 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:50.437867 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:50.438664 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:50.438689 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.439938 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.439941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:50.441518 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:50.441531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.444697 (+ 3166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.444702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:50.444947 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.445738 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:50.445763 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:50.448673 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:50.451164 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:50.451172 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:50.451195 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:50.451590 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:50.451954 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:50.451958 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:50.451960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:50.452158 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:50.452756 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:50.452764 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:50.453167 (+ 403us) client_negotiation.cc:769] Sending connection context
0809 03:36:50.454414 (+ 1247us) client_negotiation.cc:240] Negotiation successful
0809 03:36:50.454769 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3655,"thread_start_us":204,"threads_started":1}
I20260809 03:36:50.457775 20321 heartbeater.cc:347] Connected to a master server at 127.1.224.126:41595
I20260809 03:36:50.458076 20321 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:50.458796 20321 heartbeater.cc:511] Master 127.1.224.126:41595 requested a full tablet report, sending...
I20260809 03:36:50.460071 19802 ts_manager.cc:194] Registered new tserver with Master: bdc03b35ad7543f89e4e923d75538c00 (127.1.224.66:42003)
I20260809 03:36:50.460786 19802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:57491
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:36:50.475752 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.224.84:33205/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:41595
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:36:50 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
W20260809 03:36:50.663901 20332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:36:50.664317 20332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:36:50.664392 20332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:36:50.670706 20332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:36:50.670804 20332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:36:50.670913 20332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:36:50.679343 20332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.224.84:33205/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:41595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:36:50.680930 20332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:36:50.682579 20332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:36:50.698510 20341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:50.700700 20343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:36:50.701908 20340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:36:50.708596 20332 server_base.cc:1034] running on GCE node
I20260809 03:36:50.709234 20332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:36:50.710143 20332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:36:50.711333 20332 hybrid_clock.cc:648] HybridClock initialized: now 1786246610711245 us; error 107 us; skew 500 ppm
Aug 09 03:36:50 dist-test-slave-jq4t krb5kdc[19097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:36:50.715212 20332 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:36:50.716414 20332 webserver.cc:460] Webserver started at http://127.1.224.67:37215/ using document root <none> and password file <none>
I20260809 03:36:50.717196 20332 fs_manager.cc:359] Metadata directory not provided
I20260809 03:36:50.717279 20332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:36:50.717523 20332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:36:50 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:36:50.734138 20332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8"
format_stamp: "Formatted at 2026-08-09 03:36:50 on dist-test-slave-jq4t"
server_key: "e8a36bb289ea3725e5f4e806db41ce8b"
server_key_iv: "f7ad40baf453ce72176575a2a40d3fa7"
server_key_version: "kuduclusterkey@0"
I20260809 03:36:50.735170 20332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8"
format_stamp: "Formatted at 2026-08-09 03:36:50 on dist-test-slave-jq4t"
server_key: "e8a36bb289ea3725e5f4e806db41ce8b"
server_key_iv: "f7ad40baf453ce72176575a2a40d3fa7"
server_key_version: "kuduclusterkey@0"
I20260809 03:36:50.742045 20332 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 03:36:50.772465 20355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:50.774711 20332 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.001s
I20260809 03:36:50.774961 20332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8"
format_stamp: "Formatted at 2026-08-09 03:36:50 on dist-test-slave-jq4t"
server_key: "e8a36bb289ea3725e5f4e806db41ce8b"
server_key_iv: "f7ad40baf453ce72176575a2a40d3fa7"
server_key_version: "kuduclusterkey@0"
I20260809 03:36:50.775218 20332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:36:50.813971 20332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:36:50.817986 20332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:36:50.818212 20332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:36:50.818970 20332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:36:50.820150 20332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:36:50.820218 20332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:50.820276 20332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:36:50.820305 20332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:36:50.834118 20332 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:42049
I20260809 03:36:50.835608 20332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:36:50.844774 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 20332
I20260809 03:36:50.844753 20513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:42049 every 8 connection(s)
I20260809 03:36:50.844915 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:36:50.845269 1921 external_mini_cluster.cc:1428] Setting key c2894198a3c01d0fcfdec22cf16be4a1
Aug 09 03:36:50 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:36:50.854560 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.838113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:60479 (local address 127.1.224.126:41595)
0809 03:36:50.838274 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:50.838279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:50.839491 (+ 1212us) server_negotiation.cc:407] Connection header received
0809 03:36:50.840873 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:50.840878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:50.840940 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:50.841052 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:50.842568 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.843403 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.845397 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.845746 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.845838 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.850274 (+ 4436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:50.850278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:50.850281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:50.850311 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:50.852017 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:50.852711 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:50.852715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:50.852717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:50.852813 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:50.853256 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:50.853259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:50.853261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:50.853483 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:50.853576 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:50.854270 (+ 694us) server_negotiation.cc:299] Negotiation successful
0809 03:36:50.854399 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:36:50.855603 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.838662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:41595 (local address 127.1.224.67:60479)
0809 03:36:50.839307 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:50.839370 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:50.840602 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:50.841276 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:50.841286 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:50.841666 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:36:50.842374 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:50.842394 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.843585 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.843590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:50.845242 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:50.845254 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.846375 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.846379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:50.846614 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.847570 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:36:50.847601 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:36:50.850054 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:36:50.852164 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:50.852170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:50.852187 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:50.852598 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:50.852925 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:36:50.852928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:36:50.852930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:36:50.853146 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:36:50.853609 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:36:50.853617 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:36:50.853956 (+ 339us) client_negotiation.cc:769] Sending connection context
0809 03:36:50.854190 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 03:36:50.854553 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":168,"threads_started":1}
I20260809 03:36:50.856956 20514 heartbeater.cc:347] Connected to a master server at 127.1.224.126:41595
I20260809 03:36:50.857239 20514 heartbeater.cc:464] Registering TS with master...
I20260809 03:36:50.857856 20514 heartbeater.cc:511] Master 127.1.224.126:41595 requested a full tablet report, sending...
I20260809 03:36:50.859133 19802 ts_manager.cc:194] Registered new tserver with Master: 1374d93954c14c8fa2b35f0ddbcc3ae8 (127.1.224.67:42049)
I20260809 03:36:50.859738 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:36:50.859894 19802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:60479
I20260809 03:36:50.872718 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.862843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59324 (local address 127.1.224.126:41595)
0809 03:36:50.862966 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:50.862970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:50.862982 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:36:50.863019 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:50.863022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:50.863063 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:50.863139 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:36:50.864079 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.864919 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.866343 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.866666 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.866745 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.868882 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:36:50.868886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:36:50.868890 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:36:50.868919 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:36:50.870772 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:50.871303 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:50.871306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:50.871308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:50.871402 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:36:50.871720 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:36:50.871723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:36:50.871725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:36:50.871941 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:36:50.872063 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:50.872381 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 03:36:50.872555 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:36:50.876681 19802 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:36:50.881542 19802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:36:50.902312 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.892716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33033 (local address 127.0.0.1:47196)
0809 03:36:50.895437 (+ 2721us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:50.895492 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:50.895630 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:50.897115 (+ 1485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:50.897123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:50.897147 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:50.897303 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:50.897310 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.898770 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.898773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:50.900144 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:50.900156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.901747 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.901750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:50.901868 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.901907 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 03:36:50.902015 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 03:36:50.902149 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2535,"thread_start_us":110,"threads_started":1}
I20260809 03:36:50.903342 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.893466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.1.224.65:33033)
0809 03:36:50.896415 (+ 2949us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:50.896424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:50.896451 (+ 27us) server_negotiation.cc:407] Connection header received
0809 03:36:50.896552 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:50.896558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:50.896780 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:50.896926 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:50.897440 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.898639 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.900294 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.901129 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.901534 (+ 405us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.901698 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:50.903099 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0809 03:36:50.903206 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2836,"thread_start_us":94,"threads_started":1}
I20260809 03:36:50.908521 20020 tablet_service.cc:1467] Processing CreateTablet for tablet c669c242393441aab140807473845137 (DEFAULT_TABLE table=test-table [id=447d9b6427c64c60966b1c10c7b74a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:50.909546 20534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.894250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:42049 (local address 127.0.0.1:59854)
0809 03:36:50.895849 (+ 1599us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:50.895861 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:50.895932 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:50.901257 (+ 5325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:50.901261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:50.901280 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:50.901383 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:50.901389 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.905852 (+ 4463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.905856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:50.907258 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:50.907270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.908764 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.908768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:50.908917 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.908946 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 03:36:50.909339 (+ 393us) client_negotiation.cc:240] Negotiation successful
0809 03:36:50.909418 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1541,"thread_start_us":80,"threads_started":1}
I20260809 03:36:50.909771 20020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c669c242393441aab140807473845137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:50.910408 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.894846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:42003 (local address 127.0.0.1:56832)
0809 03:36:50.896097 (+ 1251us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:50.896108 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:50.896177 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:50.903250 (+ 7073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:50.903253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:50.903271 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:50.905599 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:50.905607 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.907795 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.907799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:50.908927 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:50.908939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.910028 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.910032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:50.910119 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.910145 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 03:36:50.910195 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 03:36:50.910263 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1192,"thread_start_us":86,"threads_started":1}
I20260809 03:36:50.910523 20536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.895183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59854 (local address 127.1.224.67:42049)
0809 03:36:50.900758 (+ 5575us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:50.900770 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:50.900796 (+ 26us) server_negotiation.cc:407] Connection header received
0809 03:36:50.900874 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:50.900880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:50.901102 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:50.901276 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:50.903616 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.904787 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.907562 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.908619 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.910033 (+ 1414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.910197 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:50.910249 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 03:36:50.910338 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5454,"thread_start_us":164,"threads_started":1}
I20260809 03:36:50.912211 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:50.895605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56832 (local address 127.1.224.66:42003)
0809 03:36:50.901596 (+ 5991us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:50.901701 (+ 105us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:50.901728 (+ 27us) server_negotiation.cc:407] Connection header received
0809 03:36:50.901809 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:50.901814 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:50.902835 (+ 1021us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:50.903085 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:50.905749 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.907657 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.909122 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:50.909875 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:50.911774 (+ 1899us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:50.911912 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:50.911964 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 03:36:50.912071 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5872,"thread_start_us":93,"threads_started":1}
I20260809 03:36:50.913610 20401 tablet_service.cc:1467] Processing CreateTablet for tablet c669c242393441aab140807473845137 (DEFAULT_TABLE table=test-table [id=447d9b6427c64c60966b1c10c7b74a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:50.914737 20401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c669c242393441aab140807473845137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:50.915218 20205 tablet_service.cc:1467] Processing CreateTablet for tablet c669c242393441aab140807473845137 (DEFAULT_TABLE table=test-table [id=447d9b6427c64c60966b1c10c7b74a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:36:50.928680 20205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c669c242393441aab140807473845137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:36:50.929385 20540 tablet_bootstrap.cc:492] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8: Bootstrap starting.
I20260809 03:36:50.932915 20540 tablet_bootstrap.cc:654] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:50.934111 20540 log.cc:824] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:50.939450 20540 tablet_bootstrap.cc:492] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8: No bootstrap required, opened a new log
I20260809 03:36:50.939684 20540 ts_tablet_manager.cc:1397] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.005s
I20260809 03:36:50.940943 20539 tablet_bootstrap.cc:492] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697: Bootstrap starting.
I20260809 03:36:50.943555 20539 tablet_bootstrap.cc:654] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:50.944142 20540 raft_consensus.cc:348] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35cb4370d12f4ba5a0216f4d366d0697" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33033 } } peers { permanent_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 42049 } } peers { permanent_uuid: "bdc03b35ad7543f89e4e923d75538c00" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42003 } }
I20260809 03:36:50.946044 20539 log.cc:824] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:50.947919 20540 raft_consensus.cc:374] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:50.948037 20540 raft_consensus.cc:729] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1374d93954c14c8fa2b35f0ddbcc3ae8, State: Initialized, Role: FOLLOWER
I20260809 03:36:50.948673 20540 consensus_queue.cc:260] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35cb4370d12f4ba5a0216f4d366d0697" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33033 } } peers { permanent_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 42049 } } peers { permanent_uuid: "bdc03b35ad7543f89e4e923d75538c00" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42003 } }
I20260809 03:36:50.949180 20543 tablet_bootstrap.cc:492] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00: Bootstrap starting.
I20260809 03:36:50.949821 20540 ts_tablet_manager.cc:1428] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260809 03:36:50.950263 20514 heartbeater.cc:503] Master 127.1.224.126:41595 was elected leader, sending a full tablet report...
I20260809 03:36:50.952625 20543 tablet_bootstrap.cc:654] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00: Neither blocks nor log segments found. Creating new log.
I20260809 03:36:50.959695 20543 log.cc:824] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00: Log is configured to *not* fsync() on all Append() calls
I20260809 03:36:50.965793 20543 tablet_bootstrap.cc:492] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00: No bootstrap required, opened a new log
I20260809 03:36:50.966038 20543 ts_tablet_manager.cc:1397] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.004s
I20260809 03:36:50.972015 20543 raft_consensus.cc:348] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35cb4370d12f4ba5a0216f4d366d0697" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33033 } } peers { permanent_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 42049 } } peers { permanent_uuid: "bdc03b35ad7543f89e4e923d75538c00" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42003 } }
I20260809 03:36:50.972359 20543 raft_consensus.cc:374] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:50.972426 20543 raft_consensus.cc:729] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdc03b35ad7543f89e4e923d75538c00, State: Initialized, Role: FOLLOWER
I20260809 03:36:50.974548 20543 consensus_queue.cc:260] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35cb4370d12f4ba5a0216f4d366d0697" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33033 } } peers { permanent_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 42049 } } peers { permanent_uuid: "bdc03b35ad7543f89e4e923d75538c00" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42003 } }
I20260809 03:36:50.974746 20539 tablet_bootstrap.cc:492] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697: No bootstrap required, opened a new log
I20260809 03:36:50.975045 20539 ts_tablet_manager.cc:1397] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697: Time spent bootstrapping tablet: real 0.034s user 0.005s sys 0.000s
I20260809 03:36:50.975500 20321 heartbeater.cc:503] Master 127.1.224.126:41595 was elected leader, sending a full tablet report...
I20260809 03:36:50.975948 20543 ts_tablet_manager.cc:1428] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00: Time spent starting tablet: real 0.010s user 0.001s sys 0.006s
Aug 09 03:36:50 dist-test-slave-jq4t krb5kdc[19097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246610, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:36:50.981402 20539 raft_consensus.cc:348] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35cb4370d12f4ba5a0216f4d366d0697" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33033 } } peers { permanent_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 42049 } } peers { permanent_uuid: "bdc03b35ad7543f89e4e923d75538c00" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42003 } }
I20260809 03:36:50.981841 20539 raft_consensus.cc:374] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:36:50.981948 20539 raft_consensus.cc:729] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35cb4370d12f4ba5a0216f4d366d0697, State: Initialized, Role: FOLLOWER
I20260809 03:36:50.983057 20539 consensus_queue.cc:260] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35cb4370d12f4ba5a0216f4d366d0697" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33033 } } peers { permanent_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 42049 } } peers { permanent_uuid: "bdc03b35ad7543f89e4e923d75538c00" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42003 } }
I20260809 03:36:50.984524 20539 ts_tablet_manager.cc:1428] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260809 03:36:50.985176 20120 heartbeater.cc:503] Master 127.1.224.126:41595 was elected leader, sending a full tablet report...
W20260809 03:36:51.026177 20122 tablet.cc:2369] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:36:51.075934 20551 raft_consensus.cc:482] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:36:51.076197 20551 raft_consensus.cc:504] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35cb4370d12f4ba5a0216f4d366d0697" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33033 } } peers { permanent_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 42049 } } peers { permanent_uuid: "bdc03b35ad7543f89e4e923d75538c00" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42003 } }
I20260809 03:36:51.078826 20551 leader_election.cc:288] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35cb4370d12f4ba5a0216f4d366d0697 (127.1.224.65:33033), 1374d93954c14c8fa2b35f0ddbcc3ae8 (127.1.224.67:42049)
I20260809 03:36:51.083726 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:51.078320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:43525 (local address 127.1.224.65:33033)
0809 03:36:51.078471 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:51.078478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:51.079179 (+ 701us) server_negotiation.cc:407] Connection header received
0809 03:36:51.079334 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:51.079337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:51.079404 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:51.079493 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:51.079798 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.080722 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.082321 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.083137 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.083264 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:51.083324 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:51.083510 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 03:36:51.083581 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:36:51.083743 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:51.077901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33033 (local address 127.1.224.66:43525)
0809 03:36:51.079089 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:51.079104 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:51.079234 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:51.079506 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:51.079509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:51.079543 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:51.079694 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:51.079701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.080843 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.080848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:51.082203 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:51.082215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.083264 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.083268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:51.083370 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:51.083404 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 03:36:51.083494 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 03:36:51.083582 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1116,"thread_start_us":117,"threads_started":1}
I20260809 03:36:51.084509 20039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c669c242393441aab140807473845137" candidate_uuid: "bdc03b35ad7543f89e4e923d75538c00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35cb4370d12f4ba5a0216f4d366d0697" is_pre_election: true
I20260809 03:36:51.084908 20039 raft_consensus.cc:2454] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdc03b35ad7543f89e4e923d75538c00 in term 0.
I20260809 03:36:51.085492 20163 leader_election.cc:302] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35cb4370d12f4ba5a0216f4d366d0697, bdc03b35ad7543f89e4e923d75538c00; no voters:
I20260809 03:36:51.085897 20551 raft_consensus.cc:2781] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:36:51.085999 20551 raft_consensus.cc:482] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:36:51.086045 20551 raft_consensus.cc:3037] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:51.087483 20551 raft_consensus.cc:504] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35cb4370d12f4ba5a0216f4d366d0697" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33033 } } peers { permanent_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 42049 } } peers { permanent_uuid: "bdc03b35ad7543f89e4e923d75538c00" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42003 } }
I20260809 03:36:51.088423 20039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c669c242393441aab140807473845137" candidate_uuid: "bdc03b35ad7543f89e4e923d75538c00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35cb4370d12f4ba5a0216f4d366d0697"
I20260809 03:36:51.088624 20039 raft_consensus.cc:3037] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:36:51.090039 20039 raft_consensus.cc:2454] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdc03b35ad7543f89e4e923d75538c00 in term 1.
I20260809 03:36:51.090598 20163 leader_election.cc:302] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35cb4370d12f4ba5a0216f4d366d0697, bdc03b35ad7543f89e4e923d75538c00; no voters:
I20260809 03:36:51.091198 20551 leader_election.cc:288] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [CANDIDATE]: Term 1 election: Requested vote from peers 35cb4370d12f4ba5a0216f4d366d0697 (127.1.224.65:33033), 1374d93954c14c8fa2b35f0ddbcc3ae8 (127.1.224.67:42049)
I20260809 03:36:51.091362 20551 raft_consensus.cc:2781] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:36:51.091480 20551 raft_consensus.cc:686] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 1 LEADER]: Becoming Leader. State: Replica: bdc03b35ad7543f89e4e923d75538c00, State: Running, Role: LEADER
I20260809 03:36:51.091964 20551 consensus_queue.cc:237] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35cb4370d12f4ba5a0216f4d366d0697" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33033 } } peers { permanent_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 42049 } } peers { permanent_uuid: "bdc03b35ad7543f89e4e923d75538c00" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42003 } }
I20260809 03:36:51.092456 20536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:51.079310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:58947 (local address 127.1.224.67:42049)
0809 03:36:51.079697 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:51.079702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:51.084129 (+ 4427us) server_negotiation.cc:407] Connection header received
0809 03:36:51.084384 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:51.084390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:51.084466 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:51.084558 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:36:51.084887 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.085650 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.090986 (+ 5336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.091651 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.091768 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:51.091828 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:51.092218 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 03:36:51.092317 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260809 03:36:51.092868 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:51.078587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:42049 (local address 127.1.224.66:58947)
0809 03:36:51.084018 (+ 5431us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:51.084033 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:51.084128 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:51.084648 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:36:51.084654 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:36:51.084674 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:36:51.084744 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:51.084749 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.089528 (+ 4779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.089532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:51.090760 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:36:51.090777 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.091930 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.091933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:36:51.092034 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:51.092061 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 03:36:51.092658 (+ 597us) client_negotiation.cc:240] Negotiation successful
0809 03:36:51.092749 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5374,"thread_start_us":108,"threads_started":1}
I20260809 03:36:51.093938 20447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c669c242393441aab140807473845137" candidate_uuid: "bdc03b35ad7543f89e4e923d75538c00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8" is_pre_election: true
I20260809 03:36:51.094309 20447 raft_consensus.cc:2454] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdc03b35ad7543f89e4e923d75538c00 in term 0.
I20260809 03:36:51.094700 20444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c669c242393441aab140807473845137" candidate_uuid: "bdc03b35ad7543f89e4e923d75538c00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8"
I20260809 03:36:51.094810 20444 raft_consensus.cc:3037] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8 [term 0 FOLLOWER]: Advancing to term 1
W20260809 03:36:51.098937 20515 tablet.cc:2369] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:36:51.105598 20444 raft_consensus.cc:2454] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdc03b35ad7543f89e4e923d75538c00 in term 1.
I20260809 03:36:51.115676 19801 catalog_manager.cc:5495] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdc03b35ad7543f89e4e923d75538c00 (127.1.224.66). New cstate: current_term: 1 leader_uuid: "bdc03b35ad7543f89e4e923d75538c00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35cb4370d12f4ba5a0216f4d366d0697" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 42049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdc03b35ad7543f89e4e923d75538c00" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42003 } health_report { overall_health: HEALTHY } } }
I20260809 03:36:51.174655 20537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:51.168086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56834 (local address 127.1.224.66:42003)
0809 03:36:51.168259 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 03:36:51.168266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:36:51.168288 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:36:51.168420 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:36:51.168425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:36:51.168557 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:36:51.168713 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:36:51.169297 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.170149 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.171770 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:36:51.172222 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:36:51.172353 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:36:51.173000 (+ 647us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:36:51.173761 (+ 761us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:36:51.173905 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 03:36:51.174375 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 03:36:51.174460 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:36:51.182523 20444 raft_consensus.cc:1261] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8 [term 1 FOLLOWER]: Refusing update from remote peer bdc03b35ad7543f89e4e923d75538c00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:51.183504 20551 consensus_queue.cc:1035] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1374d93954c14c8fa2b35f0ddbcc3ae8" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 42049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:36:51.183706 20325 tablet.cc:2369] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:36:51.193518 20039 raft_consensus.cc:1261] T c669c242393441aab140807473845137 P 35cb4370d12f4ba5a0216f4d366d0697 [term 1 FOLLOWER]: Refusing update from remote peer bdc03b35ad7543f89e4e923d75538c00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:36:51.194461 20551 consensus_queue.cc:1035] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35cb4370d12f4ba5a0216f4d366d0697" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:36:51.199429 20563 mvcc.cc:205] Tried to move back new op lower bound from 7316466119394742272 to 7316466119093014528. Current Snapshot: MvccSnapshot[applied={T|T < 7316466119394742272}]
I20260809 03:36:51.205912 20564 mvcc.cc:205] Tried to move back new op lower bound from 7316466119394742272 to 7316466119093014528. Current Snapshot: MvccSnapshot[applied={T|T < 7316466119394742272}]
I20260809 03:36:51.220552 20567 mvcc.cc:205] Tried to move back new op lower bound from 7316466119394742272 to 7316466119093014528. Current Snapshot: MvccSnapshot[applied={T|T < 7316466119394742272}]
I20260809 03:36:51.232205 19801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:51.232437 19801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:36:51.235710 19801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bfc88dc4345d42788e8d2a5fcb173230: Sending DeleteTablet for 3 replicas of tablet c669c242393441aab140807473845137
I20260809 03:36:51.236737 20020 tablet_service.cc:1514] Processing DeleteTablet for tablet c669c242393441aab140807473845137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:51 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:47196
I20260809 03:36:51.236904 20205 tablet_service.cc:1514] Processing DeleteTablet for tablet c669c242393441aab140807473845137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:51 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:56832
I20260809 03:36:51.237038 20401 tablet_service.cc:1514] Processing DeleteTablet for tablet c669c242393441aab140807473845137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:36:51 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:59854
I20260809 03:36:51.237987 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 19926
I20260809 03:36:51.260812 20578 tablet_replica.cc:330] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8: stopping tablet replica
I20260809 03:36:51.261355 20578 raft_consensus.cc:2229] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:36:51.261852 20578 raft_consensus.cc:2258] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:51.262379 20577 tablet_replica.cc:330] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00: stopping tablet replica
I20260809 03:36:51.262833 20577 raft_consensus.cc:2229] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:36:51.263358 20577 raft_consensus.cc:2258] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:36:51.269652 20577 ts_tablet_manager.cc:1905] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:36:51.270183 20578 ts_tablet_manager.cc:1905] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 03:36:51.271044 19776 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:33033 (error 108)
I20260809 03:36:51.273530 20578 ts_tablet_manager.cc:1918] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:51.273653 20578 log.cc:1196] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/c669c242393441aab140807473845137
I20260809 03:36:51.274011 20578 ts_tablet_manager.cc:1939] T c669c242393441aab140807473845137 P 1374d93954c14c8fa2b35f0ddbcc3ae8: Deleting consensus metadata
I20260809 03:36:51.275557 19775 catalog_manager.cc:4841] TS 1374d93954c14c8fa2b35f0ddbcc3ae8 (127.1.224.67:42049): tablet c669c242393441aab140807473845137 (table test-table [id=447d9b6427c64c60966b1c10c7b74a2d]) successfully deleted
W20260809 03:36:51.280990 20531 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:36:51.271953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33033 (local address 127.0.0.1:47208)
0809 03:36:51.272182 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:51.272200 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:36:51.272309 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:36:51.280805 (+ 8496us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:33033: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":130}
W20260809 03:36:51.281421 19776 catalog_manager.cc:4584] TS 35cb4370d12f4ba5a0216f4d366d0697 (127.1.224.65:33033): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c669c242393441aab140807473845137: Network error: Client connection negotiation failed: client connection to 127.1.224.65:33033: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:36:51.281998 20577 ts_tablet_manager.cc:1918] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:36:51.282152 20577 log.cc:1196] T c669c242393441aab140807473845137 P bdc03b35ad7543f89e4e923d75538c00: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/c669c242393441aab140807473845137
I20260809 03:36:51.283191 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 20132
I20260809 03:36:51.334985 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:51.334574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33033 (local address 127.0.0.1:47218)
0809 03:36:51.334758 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:51.334869 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 03:36:51.335296 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 20332
I20260809 03:36:51.352113 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:51.351629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33033 (local address 127.0.0.1:47232)
0809 03:36:51.351888 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:51.351966 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260809 03:36:51.353046 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:36:51.352751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:42003 (local address 127.0.0.1:56840)
0809 03:36:51.352881 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 03:36:51.352949 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.66:42003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 03:36:51.358806 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 19746
I20260809 03:36:51.373332 1921 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 03:36:51.719439 1921 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 03:36:51.719590 1921 mini_ranger.cc:66] Stopping Ranger...
I20260809 03:36:52.065897 1921 mini_ranger.cc:68] Stopped Ranger
2026-08-09 03:36:52.066 UTC [19139] LOG: received smart shutdown request
2026-08-09 03:36:52.069 UTC [19139] LOG: background worker "logical replication launcher" (PID 19150) exited with exit code 1
2026-08-09 03:36:52.070 UTC [19145] LOG: shutting down
2026-08-09 03:36:52.244 UTC [19139] LOG: database system is shut down
2026-08-09T03:36:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50247 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:36:52 dist-test-slave-jq4t krb5kdc[20593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:36:52 dist-test-slave-jq4t krb5kdc[20593](info): set up 2 sockets
Aug 09 03:36:52 dist-test-slave-jq4t krb5kdc[20593](info): commencing operation
krb5kdc: starting...
W20260809 03:36:53.649400 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:36:53 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:36:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:36:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/rangeradmin_127.1.224.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/rangeradmin_127.1.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/rangerlookup_127.1.224.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/rangerlookup_127.1.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/HTTP_127.1.224.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/HTTP_127.1.224.84@KRBTEST.COM.keytab.
I20260809 03:36:53.831314 1921 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfj0rOk/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/postgres -l logfile start
2026-08-09 03:36:56.309 UTC [20645] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 03:36:56.309 UTC [20645] LOG: listening on IPv4 address "127.1.224.84", port 53887
2026-08-09 03:36:56.312 UTC [20645] LOG: listening on Unix socket "/tmp/.s.PGSQL.53887"
2026-08-09 03:36:56.335 UTC [20651] LOG: database system was shut down at 2026-08-09 03:36:54 UTC
2026-08-09 03:36:56.338 UTC [20645] LOG: database system is ready to accept connections
I20260809 03:36:57.493963 1921 mini_postgres.cc:96] Postgres bound to 53887
2026-08-09 03:36:57.502 UTC [20662] FATAL: database "slave" does not exist
127.1.224.84:53887 - accepting connections
I20260809 03:36:57.502924 1921 mini_ranger.cc:161] Starting Ranger...
I20260809 03:36:57.520787 1921 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 03:36:58.010298 1921 mini_ranger.cc:87] Created ranger Postgres database
I20260809 03:36:58.010404 1921 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-admin
2026-08-09 03:36:58,298 [I] DB FLAVOR :POSTGRES
2026-08-09 03:36:58,299 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 03:36:58,299 [I] Checking connection..
2026-08-09 03:36:58,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 03:36:58,577 [I] Checking connection passed.
2026-08-09 03:36:58,577 [I] --------- Verifying version history table ---------
2026-08-09 03:36:58,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:36:58,830 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 03:36:58,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:36:59,101 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 03:36:59,102 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 03:36:59,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 03:36:59.351 UTC [20751] WARNING: there is no transaction in progress
2026-08-09 03:36:59,357 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 03:36:59,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:36:59,626 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 03:36:59,626 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 03:36:59,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 03:36:59,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 03:37:00,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jq4t.c.gcp-upstream.internal','N') ;"
2026-08-09 03:37:00,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 03:37:00,640 [I] Table x_portal_user does not exist in database ranger
2026-08-09 03:37:00,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 03:37:00,907 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 03:37:00,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 03:37:01,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 03:37:01,404 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 03:37:01,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 03:37:02.376 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.393 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.409 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.425 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.442 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.460 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.517 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.524 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.540 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.550 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.560 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.574 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.584 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.591 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.599 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:02.606 UTC [20896] WARNING: there is no transaction in progress
2026-08-09 03:37:03,255 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 03:37:03,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 03:37:03,521 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 03:37:03,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 03:37:03,762 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 03:37:03,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 03:37:04,036 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 03:37:04,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 03:37:04,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 03:37:04,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jq4t.c.gcp-upstream.internal';"
2026-08-09 03:37:04,816 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 03:37:04,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 03:37:05,065 [I] DB_PATCHES have already been applied
I20260809 03:37:05.071139 1921 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfj0rOk/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 03:37:05.071249 1921 mini_ranger.cc:193] Using host: 127.1.224.84
I20260809 03:37:05.075737 1921 mini_ranger.cc:227] Ranger admin URL: http://127.1.224.84:37637
Aug 09, 2026 3:37:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:37:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:37:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 3:37:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 3:37:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 3:37:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 3:37:05 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 03:37:06.461253 1921 mini_ranger.cc:160] Time spent starting Ranger: real 8.958s user 0.000s sys 0.012s
Aug 09, 2026 3:37:25 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 3:37:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 3:37:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 3:37:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 3:37:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 3:37:26 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 03:37:27.568737 1921 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/rangerkms_127.1.224.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/rangerkms_127.1.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.224.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.224.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.224.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/HTTP_127.1.224.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.224.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/HTTP_127.1.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 03:37:27 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246647, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 03:37:27.677395 1921 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 03:37:27.689986 1921 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 03:37:28.558424 1921 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 03:37:28.558511 1921 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-kms
I20260809 03:37:28.558525 1921 mini_ranger_kms.cc:227] Using postgres at 127.1.224.84:53887
2026-08-09 03:37:28,746 [I] DB FLAVOR :POSTGRES
2026-08-09 03:37:28,746 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 03:37:28,746 [I] Checking connection
2026-08-09 03:37:28,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 03:37:29,219 [I] connection success
2026-08-09 03:37:29,219 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 03:37:29,219 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 03:37:29,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 03:37:29,623 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 03:37:29,623 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 03:37:29,623 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 03:37:29,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.224.84:53887/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 03:37:29,896 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 03:37:30.384835 1921 mini_ranger_kms.cc:310] Created kms service
I20260809 03:37:30.610590 1921 mini_ranger_kms.cc:326] Created kudu user
I20260809 03:37:30.657294 1921 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 03:37:30 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246647, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:37:31.064945 1921 mini_ranger_kms.cc:384] Added ranger policy
I20260809 03:37:31.065234 1921 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfj0rOk/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfj0rOk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 03:37:31.065260 1921 mini_ranger_kms.cc:240] Using host: 127.1.224.84
I20260809 03:37:31.071136 1921 mini_ranger_kms.cc:276] Ranger KMS PID: 21166
I20260809 03:37:31.071266 1921 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.224.84:52749
Aug 09, 2026 3:37:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:37:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:37:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 3:37:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 3:37:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 3:37:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 03:37:33 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246653, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.224.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:33.845983 1921 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.169s user 0.007s sys 0.165s
I20260809 03:37:33.846134 1921 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 03:37:33.846218 1921 mini_ranger_kms.cc:401] 127.1.224.84:52749/kms/v1/keys
Aug 09 03:37:34 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246653, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.224.84@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
Aug 09 03:37:34 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246653, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.224.84@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
Aug 09 03:37:36 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:37:36.836262 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:44207
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:44207
--ranger_config_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.224.84:52749/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:37:36.961324 21236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:36.961635 21236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:36.961679 21236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:36.966051 21236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:37:36.966131 21236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:36.966152 21236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:36.966168 21236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:37:36.966183 21236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:37:36.972226 21236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.224.84:52749/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:44207
--ranger_config_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:44207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:36.973722 21236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:36.975121 21236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:36.984733 21247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:36.984759 21236 server_base.cc:1034] running on GCE node
W20260809 03:37:36.984786 21248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:36.985062 21251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:36.985750 21236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:36.987021 21236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:36.988307 21236 hybrid_clock.cc:648] HybridClock initialized: now 1786246656988272 us; error 51 us; skew 500 ppm
Aug 09 03:37:36 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246656, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:36.991658 21236 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:37:36.992913 21236 webserver.cc:460] Webserver started at http://127.1.224.126:44907/ using document root <none> and password file <none>
I20260809 03:37:36.993542 21236 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:36.993603 21236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:36.993901 21236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:37:36 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246656, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:37:37.057111 21236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "5502cfbf62f5419c8eb381c3f50554ee"
format_stamp: "Formatted at 2026-08-09 03:37:37 on dist-test-slave-jq4t"
server_key: "817c70ea6b8b2940e37e63f2c9800ad4"
server_key_iv: "6362dab792574defd0773298a3aa2761"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:37.057813 21236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "5502cfbf62f5419c8eb381c3f50554ee"
format_stamp: "Formatted at 2026-08-09 03:37:37 on dist-test-slave-jq4t"
server_key: "817c70ea6b8b2940e37e63f2c9800ad4"
server_key_iv: "6362dab792574defd0773298a3aa2761"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:37.062209 21236 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:37:37.081385 21263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:37.083261 21236 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:37:37.083456 21236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "5502cfbf62f5419c8eb381c3f50554ee"
format_stamp: "Formatted at 2026-08-09 03:37:37 on dist-test-slave-jq4t"
server_key: "817c70ea6b8b2940e37e63f2c9800ad4"
server_key_iv: "6362dab792574defd0773298a3aa2761"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:37.083632 21236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:37.099975 21236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:37.103232 21236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:37.103410 21236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:37.110872 21236 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:44207
I20260809 03:37:37.110890 21342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:44207 every 8 connection(s)
I20260809 03:37:37.111943 21236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:37:37.114974 21343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:37.116178 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 21236
I20260809 03:37:37.116287 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:37:37.116667 1921 external_mini_cluster.cc:1428] Setting key ab565ac041a1036ac95449d8e3aa20fe
I20260809 03:37:37.121531 21343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee: Bootstrap starting.
I20260809 03:37:37.123879 21343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:37.124835 21343 log.cc:824] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:37:37 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:37.127362 21343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee: No bootstrap required, opened a new log
I20260809 03:37:37.131227 21343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5502cfbf62f5419c8eb381c3f50554ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44207 } }
I20260809 03:37:37.131567 21343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:37.131660 21343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5502cfbf62f5419c8eb381c3f50554ee, State: Initialized, Role: FOLLOWER
I20260809 03:37:37.132126 21343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5502cfbf62f5419c8eb381c3f50554ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44207 } }
I20260809 03:37:37.132279 21343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:37:37.132364 21343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:37:37.132476 21343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:37.132611 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:37.118254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53212 (local address 127.1.224.126:44207)
0809 03:37:37.118742 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:37.118759 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:37.118797 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:37:37.119463 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:37.119482 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:37.119845 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:37.120203 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:37.121114 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:37.122184 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:37.123078 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:37.123576 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:37.123756 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:37.126808 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:37.126814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:37.126830 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:37.126859 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:37.129257 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:37.129799 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:37.129804 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:37.129811 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:37.129923 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:37.130231 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:37.130233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:37.130234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:37.130706 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:37.130893 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:37.131197 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 03:37:37.131513 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":159,"threads_started":1}
I20260809 03:37:37.133540 21343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5502cfbf62f5419c8eb381c3f50554ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44207 } }
I20260809 03:37:37.133934 21343 leader_election.cc:302] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5502cfbf62f5419c8eb381c3f50554ee; no voters:
I20260809 03:37:37.134306 21343 leader_election.cc:288] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:37:37.134418 21351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:37:37.134661 21351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 1 LEADER]: Becoming Leader. State: Replica: 5502cfbf62f5419c8eb381c3f50554ee, State: Running, Role: LEADER
I20260809 03:37:37.135049 21351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5502cfbf62f5419c8eb381c3f50554ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44207 } }
I20260809 03:37:37.135632 21343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:37:37.136922 21353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5502cfbf62f5419c8eb381c3f50554ee. Latest consensus state: current_term: 1 leader_uuid: "5502cfbf62f5419c8eb381c3f50554ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5502cfbf62f5419c8eb381c3f50554ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44207 } } }
I20260809 03:37:37.137080 21353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:37.137387 21354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5502cfbf62f5419c8eb381c3f50554ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5502cfbf62f5419c8eb381c3f50554ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44207 } } }
I20260809 03:37:37.137475 21354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:37.138139 21343 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 03:37:37.138478 21362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:37:37.144764 21362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:37:37.152349 21362 catalog_manager.cc:1347] Generated new cluster ID: cf70bc0fe06248338e5cfa1d7c33f739
I20260809 03:37:37.152523 21362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:37:37.201124 21362 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:37:37.202234 21362 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:37:37.208950 21362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee: Generated new TSK 0
I20260809 03:37:37.209904 21362 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:37:37.841893 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.224.84:52749/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:44207
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:37:38.006399 21400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:38.006757 21400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:38.006835 21400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:38.011312 21400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:38.011420 21400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:38.011513 21400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:37:38.018532 21400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.224.84:52749/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:44207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:38.020294 21400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:38.021790 21400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:38.033519 21418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:38.033519 21415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:38.033862 21416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:38.034090 21400 server_base.cc:1034] running on GCE node
I20260809 03:37:38.034803 21400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:38.035773 21400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:38.037096 21400 hybrid_clock.cc:648] HybridClock initialized: now 1786246658037070 us; error 43 us; skew 500 ppm
Aug 09 03:37:38 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246658, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:38.046823 21400 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:37:38.048116 21400 webserver.cc:460] Webserver started at http://127.1.224.65:33747/ using document root <none> and password file <none>
I20260809 03:37:38.048851 21400 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:38.048954 21400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:38.049211 21400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:37:38 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246658, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:37:38.067338 21400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "2376cbf128f34273848ac87f41d53942"
format_stamp: "Formatted at 2026-08-09 03:37:38 on dist-test-slave-jq4t"
server_key: "d973924689077d21f49427543a904ac4"
server_key_iv: "2a7ab56f66a11ee0cb6e6e3e67c641ab"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:38.068312 21400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "2376cbf128f34273848ac87f41d53942"
format_stamp: "Formatted at 2026-08-09 03:37:38 on dist-test-slave-jq4t"
server_key: "d973924689077d21f49427543a904ac4"
server_key_iv: "2a7ab56f66a11ee0cb6e6e3e67c641ab"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:38.073827 21400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 03:37:38.091605 21428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:38.093518 21400 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 03:37:38.093755 21400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "2376cbf128f34273848ac87f41d53942"
format_stamp: "Formatted at 2026-08-09 03:37:38 on dist-test-slave-jq4t"
server_key: "d973924689077d21f49427543a904ac4"
server_key_iv: "2a7ab56f66a11ee0cb6e6e3e67c641ab"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:38.093932 21400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:38.131138 21400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:38.135097 21400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:38.135437 21400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:38.136332 21400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:38.137691 21400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:37:38.137758 21400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:38.137815 21400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:38.137876 21400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 03:37:38.152629 21400 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:43259
I20260809 03:37:38.154027 21400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:37:38.155283 21588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:43259 every 8 connection(s)
I20260809 03:37:38.163753 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 21400
I20260809 03:37:38.163908 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:37:38.164270 1921 external_mini_cluster.cc:1428] Setting key f359b86ca32d570bdebe0d7e10ba60ee
Aug 09 03:37:38 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246658, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:38.176900 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:38.159592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:59171 (local address 127.1.224.126:44207)
0809 03:37:38.162154 (+ 2562us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:38.162161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:38.162175 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:37:38.162216 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:38.162218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:38.162262 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:38.162386 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:38.164029 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:38.164893 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:38.166291 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:38.166618 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:38.166735 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:38.170161 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:38.170167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:38.170172 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:38.170207 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:38.172376 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:38.173165 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:38.173172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:38.173177 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:38.173282 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:38.173783 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:38.173787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:38.173790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:38.174075 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:38.174183 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:38.176499 (+ 2316us) server_negotiation.cc:299] Negotiation successful
0809 03:37:38.176670 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2480,"thread_start_us":120,"threads_started":1}
I20260809 03:37:38.177937 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:38.158217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:44207 (local address 127.1.224.65:59171)
0809 03:37:38.160865 (+ 2648us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:38.160918 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:38.161760 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:38.162649 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:38.162660 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:38.163050 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:38.163792 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:38.163815 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:38.165081 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:38.165085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:38.166132 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:38.166144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:38.166944 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:38.166952 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:38.167231 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:38.168039 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:38.168068 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:38.169951 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:38.172565 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:38.172575 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:38.172597 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:38.173031 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:38.173409 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:38.173412 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:38.173414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:38.173648 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:38.174246 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:38.174256 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:38.174687 (+ 431us) client_negotiation.cc:769] Sending connection context
0809 03:37:38.174936 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 03:37:38.175381 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2377,"thread_start_us":237,"threads_started":1}
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:37:38.179879 21589 heartbeater.cc:347] Connected to a master server at 127.1.224.126:44207
I20260809 03:37:38.180181 21589 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:38.180938 21589 heartbeater.cc:511] Master 127.1.224.126:44207 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:37:38.182788 21291 ts_manager.cc:194] Registered new tserver with Master: 2376cbf128f34273848ac87f41d53942 (127.1.224.65:43259)
I20260809 03:37:38.184006 21291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:59171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:37:38.203451 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.224.84:52749/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:44207
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:37:38 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246658, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 03:37:38.373118 21596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:38.373571 21596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:38.373648 21596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:38.380056 21596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:38.380155 21596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:38.380249 21596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:37:38.388840 21596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.224.84:52749/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:44207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:38.390429 21596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:38.391908 21596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:38.405472 21603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:38.405596 21607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:38.405498 21602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:38.407150 21596 server_base.cc:1034] running on GCE node
I20260809 03:37:38.407717 21596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:38.408646 21596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:38.411952 21596 hybrid_clock.cc:648] HybridClock initialized: now 1786246658411855 us; error 97 us; skew 500 ppm
Aug 09 03:37:38 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246658, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:38.416433 21596 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:37:38.417832 21596 webserver.cc:460] Webserver started at http://127.1.224.66:46825/ using document root <none> and password file <none>
I20260809 03:37:38.418691 21596 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:38.418859 21596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:38.419188 21596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:37:38 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246658, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:37:38.444725 21596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "22fa246ce82744b299ca9e6fc766287a"
format_stamp: "Formatted at 2026-08-09 03:37:38 on dist-test-slave-jq4t"
server_key: "50853e909e7b9f687641cea7c71b02f2"
server_key_iv: "50a34273e3d94251248dbe58d37f8d98"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:38.445659 21596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "22fa246ce82744b299ca9e6fc766287a"
format_stamp: "Formatted at 2026-08-09 03:37:38 on dist-test-slave-jq4t"
server_key: "50853e909e7b9f687641cea7c71b02f2"
server_key_iv: "50a34273e3d94251248dbe58d37f8d98"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:38.451285 21596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 03:37:38.469100 21618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:38.471140 21596 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:37:38.471333 21596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "22fa246ce82744b299ca9e6fc766287a"
format_stamp: "Formatted at 2026-08-09 03:37:38 on dist-test-slave-jq4t"
server_key: "50853e909e7b9f687641cea7c71b02f2"
server_key_iv: "50a34273e3d94251248dbe58d37f8d98"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:38.471522 21596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:38.499799 21596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:38.504168 21596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:38.504467 21596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:38.505383 21596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:38.506637 21596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:37:38.506758 21596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:38.506866 21596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:38.506933 21596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:38.521727 21596 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:43381
I20260809 03:37:38.523267 21596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:37:38.526026 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 21596
I20260809 03:37:38.526165 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:37:38.526520 1921 external_mini_cluster.cc:1428] Setting key 7aaf14bab451b5425c6be48ded3128d8
I20260809 03:37:38.525041 21778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:43381 every 8 connection(s)
Aug 09 03:37:38 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246658, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:38.546207 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:38.528722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:55263 (local address 127.1.224.126:44207)
0809 03:37:38.531102 (+ 2380us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:38.531108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:38.531130 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:37:38.532051 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:38.532055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:38.532114 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:38.532167 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:38.534030 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:38.534869 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:38.536996 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:38.537433 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:38.537543 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:38.541480 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:38.541485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:38.541488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:38.541533 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:38.543421 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:38.544146 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:38.544150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:38.544152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:38.544247 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:38.544740 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:38.544744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:38.544746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:38.544999 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:38.545091 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:38.545875 (+ 784us) server_negotiation.cc:299] Negotiation successful
0809 03:37:38.546027 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2271}
I20260809 03:37:38.548565 21782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:38.528938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:44207 (local address 127.1.224.66:55263)
0809 03:37:38.530539 (+ 1601us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:38.530610 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:38.531749 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:38.532663 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:38.532674 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:38.533062 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:38.533828 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:38.533852 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:38.535058 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:38.535062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:38.536829 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:38.536845 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:38.537877 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:38.537883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:38.538140 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:38.538962 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:38.538987 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:38.541214 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:38.543600 (+ 2386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:38.543608 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:38.543629 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:38.544014 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:38.544364 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:38.544368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:38.544370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:38.544621 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:38.545166 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:38.545174 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:38.545566 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 03:37:38.546804 (+ 1238us) client_negotiation.cc:240] Negotiation successful
0809 03:37:38.547205 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1306,"thread_start_us":190,"threads_started":1}
I20260809 03:37:38.550485 21780 heartbeater.cc:347] Connected to a master server at 127.1.224.126:44207
I20260809 03:37:38.550845 21780 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:38.551543 21780 heartbeater.cc:511] Master 127.1.224.126:44207 requested a full tablet report, sending...
I20260809 03:37:38.552911 21291 ts_manager.cc:194] Registered new tserver with Master: 22fa246ce82744b299ca9e6fc766287a (127.1.224.66:43381)
I20260809 03:37:38.553711 21291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:55263
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:37:38.599020 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.224.84:52749/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:44207
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:37:38 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246658, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
W20260809 03:37:38.823262 21788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:38.824162 21788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:38.824301 21788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:38.830689 21788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:38.830871 21788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:38.831029 21788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:37:38.839612 21788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.224.84:52749/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:44207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:38.841542 21788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:38.843096 21788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:38.857180 21798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:38.857184 21796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:38.859282 21788 server_base.cc:1034] running on GCE node
W20260809 03:37:38.866843 21795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:38.867695 21788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:38.868665 21788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:38.876128 21788 hybrid_clock.cc:648] HybridClock initialized: now 1786246658876018 us; error 120 us; skew 500 ppm
Aug 09 03:37:38 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246658, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:38.880378 21788 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:37:38.881768 21788 webserver.cc:460] Webserver started at http://127.1.224.67:41067/ using document root <none> and password file <none>
I20260809 03:37:38.882609 21788 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:38.882766 21788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:38.883073 21788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:37:38 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246658, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:37:38.899354 21788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "596dc42adfce45f1a6a8d1ff272b6917"
format_stamp: "Formatted at 2026-08-09 03:37:38 on dist-test-slave-jq4t"
server_key: "4b0c809fbe02ff179a23b03299a282ad"
server_key_iv: "4a007f6c193a69db6998a35c08c3863b"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:38.900313 21788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "596dc42adfce45f1a6a8d1ff272b6917"
format_stamp: "Formatted at 2026-08-09 03:37:38 on dist-test-slave-jq4t"
server_key: "4b0c809fbe02ff179a23b03299a282ad"
server_key_iv: "4a007f6c193a69db6998a35c08c3863b"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:38.906890 21788 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260809 03:37:38.923794 21810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:38.925545 21788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:37:38.925725 21788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "596dc42adfce45f1a6a8d1ff272b6917"
format_stamp: "Formatted at 2026-08-09 03:37:38 on dist-test-slave-jq4t"
server_key: "4b0c809fbe02ff179a23b03299a282ad"
server_key_iv: "4a007f6c193a69db6998a35c08c3863b"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:38.925886 21788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:38.951079 21788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:38.955219 21788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:38.955509 21788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:38.956295 21788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:38.957510 21788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:37:38.957635 21788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:38.957739 21788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:38.957808 21788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:38.972756 21788 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:44979
I20260809 03:37:38.974135 21788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:37:38.975879 21968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:44979 every 8 connection(s)
I20260809 03:37:38.977702 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 21788
I20260809 03:37:38.977846 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:37:38.978181 1921 external_mini_cluster.cc:1428] Setting key 6126aab59428d53db0099a18b388a887
Aug 09 03:37:38 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246658, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:38.995263 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:38.980125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:45375 (local address 127.1.224.126:44207)
0809 03:37:38.980254 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:38.980259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:38.980820 (+ 561us) server_negotiation.cc:407] Connection header received
0809 03:37:38.982138 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:38.982141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:38.982198 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:38.982248 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:38.983995 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:38.984826 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:38.986713 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:38.987100 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:38.987203 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:38.990811 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:38.990816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:38.990818 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:38.990873 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:38.992679 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:38.993336 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:38.993340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:38.993343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:38.993445 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:38.993872 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:38.993876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:38.993878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:38.994122 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:38.994194 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:38.994966 (+ 772us) server_negotiation.cc:299] Negotiation successful
0809 03:37:38.995099 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:37:38.997079 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:38.978880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:44207 (local address 127.1.224.67:45375)
0809 03:37:38.980662 (+ 1782us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:38.980726 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:38.981880 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:38.982715 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:38.982724 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:38.983092 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:38.983820 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:38.983842 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:38.984998 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:38.985001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:38.986530 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:38.986544 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:38.987458 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:38.987462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:38.987696 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:38.988570 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:38.988597 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:38.990581 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:38.992817 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:38.992824 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:38.992846 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:38.993219 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:38.993555 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:38.993559 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:38.993561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:38.993766 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:38.994277 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:38.994285 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:38.994667 (+ 382us) client_negotiation.cc:769] Sending connection context
0809 03:37:38.995515 (+ 848us) client_negotiation.cc:240] Negotiation successful
0809 03:37:38.995908 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1518,"thread_start_us":168,"threads_started":1}
I20260809 03:37:38.998816 21970 heartbeater.cc:347] Connected to a master server at 127.1.224.126:44207
I20260809 03:37:38.999116 21970 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:38.999760 21970 heartbeater.cc:511] Master 127.1.224.126:44207 requested a full tablet report, sending...
I20260809 03:37:39.001130 21291 ts_manager.cc:194] Registered new tserver with Master: 596dc42adfce45f1a6a8d1ff272b6917 (127.1.224.67:44979)
I20260809 03:37:39.001886 21291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:45375
I20260809 03:37:39.004208 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:37:39.005163 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 21400
I20260809 03:37:39.044955 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 21596
I20260809 03:37:39.057618 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 21788
I20260809 03:37:39.117575 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 21236
I20260809 03:37:39.136432 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:44207
--webserver_interface=127.1.224.126
--webserver_port=44907
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:44207
--ranger_config_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.224.84:52749/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.224.84:52749/kms with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:37:39.255860 21974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:39.256155 21974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:39.256196 21974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:39.260344 21974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:37:39.260421 21974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:39.260442 21974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:39.260458 21974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:37:39.260496 21974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:37:39.266386 21974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.224.84:52749/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:44207
--ranger_config_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:44207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=44907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:39.267668 21974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:39.268786 21974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:39.277719 21982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:39.277716 21979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:39.277864 21974 server_base.cc:1034] running on GCE node
W20260809 03:37:39.278069 21980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:39.278688 21974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:39.279968 21974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:39.281157 21974 hybrid_clock.cc:648] HybridClock initialized: now 1786246659281125 us; error 50 us; skew 500 ppm
Aug 09 03:37:39 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246659, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:39.283932 21974 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:37:39.284947 21974 webserver.cc:460] Webserver started at http://127.1.224.126:44907/ using document root <none> and password file <none>
I20260809 03:37:39.285518 21974 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:39.285617 21974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 03:37:39 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246659, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:37:39.327966 21974 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:37:39.330461 21992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:39.331801 21974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:37:39.331959 21974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "5502cfbf62f5419c8eb381c3f50554ee"
format_stamp: "Formatted at 2026-08-09 03:37:37 on dist-test-slave-jq4t"
server_key: "817c70ea6b8b2940e37e63f2c9800ad4"
server_key_iv: "6362dab792574defd0773298a3aa2761"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:39.332590 21974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:39.348867 21974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:39.351622 21974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:39.351837 21974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:39.359586 21974 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:44207
I20260809 03:37:39.359629 22066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:44207 every 8 connection(s)
I20260809 03:37:39.360813 21974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:37:39.364926 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 21974
I20260809 03:37:39.365388 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.224.84:52749/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:43259
--local_ip_for_outbound_sockets=127.1.224.65
--tserver_master_addrs=127.1.224.126:44207
--webserver_port=33747
--webserver_interface=127.1.224.65
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.224.84:52749/kms with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:37:39.365531 22069 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:37:39.367372 22069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee: Bootstrap starting.
I20260809 03:37:39.386298 22069 log.cc:824] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:39.396782 22069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:37:39.397367 22069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee: Bootstrap complete.
I20260809 03:37:39.402258 22069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5502cfbf62f5419c8eb381c3f50554ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44207 } }
I20260809 03:37:39.402791 22069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5502cfbf62f5419c8eb381c3f50554ee, State: Initialized, Role: FOLLOWER
I20260809 03:37:39.403331 22069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5502cfbf62f5419c8eb381c3f50554ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44207 } }
I20260809 03:37:39.403482 22069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:37:39.403579 22069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:37:39.403697 22069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:37:39.405838 22069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5502cfbf62f5419c8eb381c3f50554ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44207 } }
I20260809 03:37:39.406204 22069 leader_election.cc:302] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5502cfbf62f5419c8eb381c3f50554ee; no voters:
I20260809 03:37:39.406548 22069 leader_election.cc:288] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 03:37:39.406721 22074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:37:39.406980 22074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [term 2 LEADER]: Becoming Leader. State: Replica: 5502cfbf62f5419c8eb381c3f50554ee, State: Running, Role: LEADER
I20260809 03:37:39.407387 22074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5502cfbf62f5419c8eb381c3f50554ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44207 } }
I20260809 03:37:39.407904 22069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:37:39.408449 22074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5502cfbf62f5419c8eb381c3f50554ee. Latest consensus state: current_term: 2 leader_uuid: "5502cfbf62f5419c8eb381c3f50554ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5502cfbf62f5419c8eb381c3f50554ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44207 } } }
I20260809 03:37:39.409180 22074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:39.409061 22075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5502cfbf62f5419c8eb381c3f50554ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5502cfbf62f5419c8eb381c3f50554ee" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44207 } } }
I20260809 03:37:39.409965 22075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:39.410085 22084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:37:39.410421 22069 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 03:37:39.417318 22084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:37:39.418140 22084 catalog_manager.cc:1259] Loaded cluster ID: cf70bc0fe06248338e5cfa1d7c33f739
I20260809 03:37:39.418228 22084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:37:39.420848 22084 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:37:39.421550 22084 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee: Loaded TSK: 0
I20260809 03:37:39.423143 22084 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:37:39.533409 22071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:39.533828 22071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:39.533905 22071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:39.540691 22071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:39.540793 22071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:39.540899 22071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:37:39.549494 22071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.224.84:52749/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:43259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=33747
--tserver_master_addrs=127.1.224.126:44207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:39.551183 22071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:39.552819 22071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:39.573292 22128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:39.573419 22131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:39.573292 22129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:39.574906 22071 server_base.cc:1034] running on GCE node
I20260809 03:37:39.575510 22071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:39.576426 22071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:39.577646 22071 hybrid_clock.cc:648] HybridClock initialized: now 1786246659577610 us; error 58 us; skew 500 ppm
Aug 09 03:37:39 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246659, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:39.581032 22071 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:37:39.582393 22071 webserver.cc:460] Webserver started at http://127.1.224.65:33747/ using document root <none> and password file <none>
I20260809 03:37:39.583245 22071 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:39.583439 22071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 03:37:39 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246659, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:37:39.606141 22071 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:37:39.609714 22141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:39.611567 22071 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:37:39.611712 22071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "2376cbf128f34273848ac87f41d53942"
format_stamp: "Formatted at 2026-08-09 03:37:38 on dist-test-slave-jq4t"
server_key: "d973924689077d21f49427543a904ac4"
server_key_iv: "2a7ab56f66a11ee0cb6e6e3e67c641ab"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:39.612526 22071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:39.649607 22071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:39.653407 22071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:39.653720 22071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:39.654574 22071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:39.655741 22071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:37:39.655848 22071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:39.655941 22071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:39.656262 22071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:39.671525 22071 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:43259
I20260809 03:37:39.671733 22302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:43259 every 8 connection(s)
I20260809 03:37:39.673002 22071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:37:39.678817 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 22071
I20260809 03:37:39.679356 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.224.84:52749/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:43381
--local_ip_for_outbound_sockets=127.1.224.66
--tserver_master_addrs=127.1.224.126:44207
--webserver_port=46825
--webserver_interface=127.1.224.66
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.224.84:52749/kms with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:37:39 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246659, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:39.704094 22309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:39.684704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:44207 (local address 127.1.224.65:55389)
0809 03:37:39.685574 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:39.685639 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:39.686753 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:39.688305 (+ 1552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:39.688314 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:39.688716 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:39.689378 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:39.689398 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:39.691558 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:39.691564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:39.693215 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:39.693231 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:39.694604 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:39.694608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:39.694796 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:39.695551 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:39.695576 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:39.697446 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:39.700270 (+ 2824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:39.700276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:39.700292 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:39.700676 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:39.701076 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:39.701079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:39.701081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:39.701246 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:39.702051 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:39.702058 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:39.702372 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 03:37:39.702609 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 03:37:39.702994 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":148,"threads_started":1}
I20260809 03:37:39.706115 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:39.684653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:55389 (local address 127.1.224.126:44207)
0809 03:37:39.685233 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:39.685254 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:39.685797 (+ 543us) server_negotiation.cc:407] Connection header received
0809 03:37:39.687301 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:39.687321 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:39.687790 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:39.688200 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:39.689592 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:39.691353 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:39.693388 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:39.694174 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:39.694358 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:39.697692 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:39.697699 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:39.697717 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:39.697750 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:39.700125 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:39.700818 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:39.700826 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:39.700831 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:39.700969 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:39.701356 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:39.701359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:39.701361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:39.701863 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:39.701994 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:39.704852 (+ 2858us) server_negotiation.cc:299] Negotiation successful
0809 03:37:39.705168 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":137,"threads_started":1}
I20260809 03:37:39.708698 22303 heartbeater.cc:347] Connected to a master server at 127.1.224.126:44207
I20260809 03:37:39.708997 22303 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:39.709672 22303 heartbeater.cc:511] Master 127.1.224.126:44207 requested a full tablet report, sending...
W20260809 03:37:39.712209 22303 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:44207 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 03:37:39.712347 22303 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:39.712605 22303 heartbeater.cc:511] Master 127.1.224.126:44207 requested a full tablet report, sending...
I20260809 03:37:39.713668 22303 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:39.713869 22303 heartbeater.cc:511] Master 127.1.224.126:44207 requested a full tablet report, sending...
W20260809 03:37:39.714725 22303 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:37:39.836761 22306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:39.837180 22306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:39.837347 22306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:39.843226 22306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:39.843400 22306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:39.843568 22306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:37:39.851222 22306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.224.84:52749/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:43381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=46825
--tserver_master_addrs=127.1.224.126:44207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:39.853132 22306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:39.854458 22306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:39.866232 22315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:39.866240 22317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:39.866354 22319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:39.868144 22306 server_base.cc:1034] running on GCE node
I20260809 03:37:39.868759 22306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:39.869666 22306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:39.870858 22306 hybrid_clock.cc:648] HybridClock initialized: now 1786246659870847 us; error 30 us; skew 500 ppm
Aug 09 03:37:39 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246659, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:39.874112 22306 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:37:39.875283 22306 webserver.cc:460] Webserver started at http://127.1.224.66:46825/ using document root <none> and password file <none>
I20260809 03:37:39.875913 22306 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:39.876022 22306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 03:37:39 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246659, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:37:39.902109 22306 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:37:39.905009 22329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:39.906572 22306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:37:39.906730 22306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "22fa246ce82744b299ca9e6fc766287a"
format_stamp: "Formatted at 2026-08-09 03:37:38 on dist-test-slave-jq4t"
server_key: "50853e909e7b9f687641cea7c71b02f2"
server_key_iv: "50a34273e3d94251248dbe58d37f8d98"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:39.907346 22306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:39.941217 22306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:39.944741 22306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:39.944967 22306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:39.945775 22306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:39.946993 22306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:37:39.947116 22306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 03:37:39.947206 22306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:39.947240 22306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:39.959803 22306 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:43381
I20260809 03:37:39.961148 22306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:37:39.961619 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 22306
I20260809 03:37:39.959829 22478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:43381 every 8 connection(s)
I20260809 03:37:39.962232 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.224.84:52749/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:44979
--local_ip_for_outbound_sockets=127.1.224.67
--tserver_master_addrs=127.1.224.126:44207
--webserver_port=41067
--webserver_interface=127.1.224.67
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.224.84:52749/kms with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:37:39 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246659, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:39.981161 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:39.966705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:48919 (local address 127.1.224.126:44207)
0809 03:37:39.966848 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:39.966853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:39.966876 (+ 23us) server_negotiation.cc:407] Connection header received
0809 03:37:39.967506 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:39.967513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:39.967595 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:39.967660 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:39.969730 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:39.970283 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:39.971564 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:39.971971 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:39.972093 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:39.975630 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:39.975637 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:39.975642 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:39.975681 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:39.977783 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:39.978592 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:39.978599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:39.978603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:39.978712 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:39.979274 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:39.979281 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:39.979283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:39.979600 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:39.979691 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:39.980687 (+ 996us) server_negotiation.cc:299] Negotiation successful
0809 03:37:39.980877 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:37:39.983321 22486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:39.965427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:44207 (local address 127.1.224.66:48919)
0809 03:37:39.966033 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:39.966099 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:39.967196 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:39.968256 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:39.968268 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:39.968710 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:39.969511 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:39.969535 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:39.970450 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:39.970454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:39.971415 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:39.971425 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:39.972306 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:39.972312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:39.972603 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:39.973407 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:39.973436 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:39.975356 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:39.977953 (+ 2597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:39.977961 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:39.977986 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:39.978442 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:39.978861 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:39.978866 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:39.978870 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:39.979139 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:39.979794 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:39.979805 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:39.980268 (+ 463us) client_negotiation.cc:769] Sending connection context
0809 03:37:39.981475 (+ 1207us) client_negotiation.cc:240] Negotiation successful
0809 03:37:39.981945 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":189,"threads_started":1}
I20260809 03:37:39.985014 22479 heartbeater.cc:347] Connected to a master server at 127.1.224.126:44207
I20260809 03:37:39.985288 22479 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:39.985925 22479 heartbeater.cc:511] Master 127.1.224.126:44207 requested a full tablet report, sending...
W20260809 03:37:39.987838 22479 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:44207 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 03:37:39.987967 22479 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:39.988168 22479 heartbeater.cc:511] Master 127.1.224.126:44207 requested a full tablet report, sending...
I20260809 03:37:39.989118 22479 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:39.989295 22479 heartbeater.cc:511] Master 127.1.224.126:44207 requested a full tablet report, sending...
W20260809 03:37:39.990061 22479 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:37:40.121850 22483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:40.122298 22483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:40.122402 22483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:40.126879 22483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:40.126977 22483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:40.127063 22483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:37:40.134562 22483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.224.84:52749/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:44979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=41067
--tserver_master_addrs=127.1.224.126:44207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:40.135859 22483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:40.137113 22483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:40.149066 22499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:40.149379 22493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:40.149807 22483 server_base.cc:1034] running on GCE node
W20260809 03:37:40.149960 22494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:40.150488 22483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:40.151271 22483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:40.152457 22483 hybrid_clock.cc:648] HybridClock initialized: now 1786246660152439 us; error 36 us; skew 500 ppm
Aug 09 03:37:40 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:40.156370 22483 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:37:40.157670 22483 webserver.cc:460] Webserver started at http://127.1.224.67:41067/ using document root <none> and password file <none>
I20260809 03:37:40.158424 22483 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:40.158630 22483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 03:37:40 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:37:40.193852 22483 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260809 03:37:40.196820 22510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:40.198343 22483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:37:40.198520 22483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "596dc42adfce45f1a6a8d1ff272b6917"
format_stamp: "Formatted at 2026-08-09 03:37:38 on dist-test-slave-jq4t"
server_key: "4b0c809fbe02ff179a23b03299a282ad"
server_key_iv: "4a007f6c193a69db6998a35c08c3863b"
server_key_version: "kuduclusterkey@0"
I20260809 03:37:40.199164 22483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:40.215193 22483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:40.218961 22483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:40.219190 22483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:40.220050 22483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:40.221367 22483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:37:40.221495 22483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:40.221634 22483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:40.221783 22483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:40.233825 22483 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:44979
I20260809 03:37:40.233903 22673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:44979 every 8 connection(s)
I20260809 03:37:40.235026 22483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:37:40.240785 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 22483
Aug 09 03:37:40 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:40.251416 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:40.237165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:55379 (local address 127.1.224.126:44207)
0809 03:37:40.237310 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:40.237316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:40.238405 (+ 1089us) server_negotiation.cc:407] Connection header received
0809 03:37:40.239475 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:40.239479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:40.239551 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:40.239635 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:40.241193 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:40.241914 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:40.243351 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:40.243593 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:40.243709 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:40.246806 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:40.246812 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:40.246819 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:40.246856 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:40.248719 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:40.249408 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:40.249412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:40.249414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:40.249551 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:40.250026 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:40.250032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:40.250035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:40.250297 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:40.250382 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:40.251103 (+ 721us) server_negotiation.cc:299] Negotiation successful
0809 03:37:40.251239 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:37:40.253474 22677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:40.237680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:44207 (local address 127.1.224.67:55379)
0809 03:37:40.238222 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:40.238282 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:40.239245 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:40.239847 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:40.239854 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:40.240167 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:40.240973 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:40.241000 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:40.242077 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:40.242079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:40.243210 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:40.243221 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:40.243768 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:40.243772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:40.243961 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:40.244688 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:40.244707 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:40.246552 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:40.248846 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:40.248855 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:40.248877 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:40.249286 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:40.249643 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:40.249646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:40.249649 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:40.249890 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:40.250480 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:40.250488 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:40.250843 (+ 355us) client_negotiation.cc:769] Sending connection context
0809 03:37:40.251068 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 03:37:40.251436 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":146,"threads_started":1}
I20260809 03:37:40.254956 22674 heartbeater.cc:347] Connected to a master server at 127.1.224.126:44207
I20260809 03:37:40.255256 22674 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:40.255379 22678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:40.241704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53224 (local address 127.1.224.126:44207)
0809 03:37:40.242588 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:40.242597 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:40.242612 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:37:40.242654 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:40.242656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:40.242710 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:40.242805 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:40.243869 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:40.244818 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:40.245922 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:40.246223 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:40.246331 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:40.248703 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:40.248708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:40.248710 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:40.248733 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:40.250487 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:40.252339 (+ 1852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:40.252344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:40.252346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:40.252436 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:40.252781 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:40.252785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:40.252786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:40.253006 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:40.253201 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:40.255066 (+ 1865us) server_negotiation.cc:299] Negotiation successful
0809 03:37:40.255208 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":119,"threads_started":1}
I20260809 03:37:40.255892 22674 heartbeater.cc:511] Master 127.1.224.126:44207 requested a full tablet report, sending...
I20260809 03:37:40.257509 22016 ts_manager.cc:194] Registered new tserver with Master: 596dc42adfce45f1a6a8d1ff272b6917 (127.1.224.67:44979)
I20260809 03:37:40.259181 22016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:55379
Aug 09 03:37:40 dist-test-slave-jq4t krb5kdc[20593](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:40.715011 22303 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:40.715371 22303 heartbeater.cc:511] Master 127.1.224.126:44207 requested a full tablet report, sending...
I20260809 03:37:40.716346 22016 ts_manager.cc:194] Registered new tserver with Master: 2376cbf128f34273848ac87f41d53942 (127.1.224.65:43259)
I20260809 03:37:40.717250 22016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:55389
Aug 09 03:37:40 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:37:40.990321 22479 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:40.990662 22479 heartbeater.cc:511] Master 127.1.224.126:44207 requested a full tablet report, sending...
I20260809 03:37:40.991484 22016 ts_manager.cc:194] Registered new tserver with Master: 22fa246ce82744b299ca9e6fc766287a (127.1.224.66:43381)
I20260809 03:37:40.992249 22016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:48919
I20260809 03:37:41.003995 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:37:41.023947 22690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.006046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53232 (local address 127.1.224.126:44207)
0809 03:37:41.006738 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:41.006743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:41.007364 (+ 621us) server_negotiation.cc:407] Connection header received
0809 03:37:41.007423 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:41.007426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:41.007779 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:41.008154 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:41.008777 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.009677 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.013555 (+ 3878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.013899 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.014012 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:41.016002 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:41.016006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:41.016009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:41.016041 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:41.017966 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:41.019755 (+ 1789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:41.019759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:41.019761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:41.019878 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:41.022296 (+ 2418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:41.022300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:41.022302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:41.022594 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:41.022737 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:41.023528 (+ 791us) server_negotiation.cc:299] Negotiation successful
0809 03:37:41.023684 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":99,"threads_started":1}
I20260809 03:37:41.029292 22016 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:37:41.034605 22016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:37:41.057154 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.047791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:43259 (local address 127.0.0.1:36324)
0809 03:37:41.049527 (+ 1736us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:41.049536 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:41.049630 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:41.050307 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:41.050313 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:41.050336 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:41.050485 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:41.050493 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.052910 (+ 2417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.052915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:41.055128 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:41.055140 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.056565 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.056568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:41.056680 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:41.056726 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 03:37:41.056846 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 03:37:41.056981 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1699,"thread_start_us":105,"threads_started":1}
I20260809 03:37:41.059011 22702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.048214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36324 (local address 127.1.224.65:43259)
0809 03:37:41.049224 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:41.049229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:41.049628 (+ 399us) server_negotiation.cc:407] Connection header received
0809 03:37:41.049747 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:41.049753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:41.049974 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:41.050116 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:41.051463 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.052761 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.055270 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.056152 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.056279 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:41.056503 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:41.057545 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0809 03:37:41.057665 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":916,"thread_start_us":95,"threads_started":1}
I20260809 03:37:41.059252 22698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.046755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:43381 (local address 127.0.0.1:35652)
0809 03:37:41.049330 (+ 2575us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:41.049340 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:41.049411 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:41.050427 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:41.050430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:41.050445 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:41.050515 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:41.050524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.054225 (+ 3701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.054228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:41.055311 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:41.055320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.058812 (+ 3492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.058816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:41.058926 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:41.058956 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 03:37:41.059012 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 03:37:41.059099 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2526,"thread_start_us":80,"threads_started":1}
I20260809 03:37:41.060949 22699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.046938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.1.224.66:43381)
0809 03:37:41.049889 (+ 2951us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:41.049896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:41.049916 (+ 20us) server_negotiation.cc:407] Connection header received
0809 03:37:41.049985 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:41.049989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:41.050155 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:41.050288 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:41.052961 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.054087 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.055460 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.056193 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.060430 (+ 4237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:41.060639 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:41.060696 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 03:37:41.060798 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2834,"thread_start_us":118,"threads_started":1}
I20260809 03:37:41.061683 22701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.046644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:44979 (local address 127.0.0.1:38148)
0809 03:37:41.048838 (+ 2194us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:41.048886 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:41.048994 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:41.051282 (+ 2288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:41.051284 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:41.051303 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:41.051432 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:41.051441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.056349 (+ 4908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.056353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:41.057425 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:41.057437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.061287 (+ 3850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.061292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:41.061398 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:41.061422 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:37:41.061473 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 03:37:41.061542 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2042,"thread_start_us":144,"threads_started":1}
I20260809 03:37:41.062644 22700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.047566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38148 (local address 127.1.224.67:44979)
0809 03:37:41.050791 (+ 3225us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:41.050799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:41.050824 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:37:41.050932 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:41.050937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:41.051123 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:41.051599 (+ 476us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:41.051739 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.052820 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.057599 (+ 4779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.058453 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.062173 (+ 3720us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:41.062338 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:41.062396 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 03:37:41.062498 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3109,"thread_start_us":103,"threads_started":1}
I20260809 03:37:41.062851 22214 tablet_service.cc:1467] Processing CreateTablet for tablet d4962572e5ba424e9a9cf39f54ea5d56 (DEFAULT_TABLE table=test-table [id=075261b4e2c64983ac2dab6641bdc5d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:37:41.064177 22214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4962572e5ba424e9a9cf39f54ea5d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:41.066254 22391 tablet_service.cc:1467] Processing CreateTablet for tablet d4962572e5ba424e9a9cf39f54ea5d56 (DEFAULT_TABLE table=test-table [id=075261b4e2c64983ac2dab6641bdc5d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:37:41.066772 22569 tablet_service.cc:1467] Processing CreateTablet for tablet d4962572e5ba424e9a9cf39f54ea5d56 (DEFAULT_TABLE table=test-table [id=075261b4e2c64983ac2dab6641bdc5d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:37:41.067432 22391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4962572e5ba424e9a9cf39f54ea5d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:41.067803 22569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4962572e5ba424e9a9cf39f54ea5d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:41.077316 22707 tablet_bootstrap.cc:492] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917: Bootstrap starting.
I20260809 03:37:41.079526 22707 tablet_bootstrap.cc:654] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:41.080111 22705 tablet_bootstrap.cc:492] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942: Bootstrap starting.
I20260809 03:37:41.080575 22707 log.cc:824] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:41.083181 22707 tablet_bootstrap.cc:492] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917: No bootstrap required, opened a new log
I20260809 03:37:41.083401 22707 ts_tablet_manager.cc:1397] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:37:41.082882 22705 tablet_bootstrap.cc:654] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:41.084918 22706 tablet_bootstrap.cc:492] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a: Bootstrap starting.
I20260809 03:37:41.085210 22705 log.cc:824] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:41.087464 22706 tablet_bootstrap.cc:654] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:41.088629 22705 tablet_bootstrap.cc:492] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942: No bootstrap required, opened a new log
I20260809 03:37:41.088873 22705 ts_tablet_manager.cc:1397] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 03:37:41.088718 22707 raft_consensus.cc:348] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596dc42adfce45f1a6a8d1ff272b6917" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44979 } } peers { permanent_uuid: "22fa246ce82744b299ca9e6fc766287a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 43381 } } peers { permanent_uuid: "2376cbf128f34273848ac87f41d53942" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43259 } }
I20260809 03:37:41.089193 22707 raft_consensus.cc:374] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:41.089299 22707 raft_consensus.cc:729] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 596dc42adfce45f1a6a8d1ff272b6917, State: Initialized, Role: FOLLOWER
I20260809 03:37:41.089936 22707 consensus_queue.cc:260] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596dc42adfce45f1a6a8d1ff272b6917" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44979 } } peers { permanent_uuid: "22fa246ce82744b299ca9e6fc766287a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 43381 } } peers { permanent_uuid: "2376cbf128f34273848ac87f41d53942" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43259 } }
I20260809 03:37:41.091343 22707 ts_tablet_manager.cc:1428] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 03:37:41.091853 22674 heartbeater.cc:503] Master 127.1.224.126:44207 was elected leader, sending a full tablet report...
I20260809 03:37:41.093618 22705 raft_consensus.cc:348] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596dc42adfce45f1a6a8d1ff272b6917" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44979 } } peers { permanent_uuid: "22fa246ce82744b299ca9e6fc766287a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 43381 } } peers { permanent_uuid: "2376cbf128f34273848ac87f41d53942" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43259 } }
I20260809 03:37:41.094024 22705 raft_consensus.cc:374] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:41.094122 22705 raft_consensus.cc:729] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2376cbf128f34273848ac87f41d53942, State: Initialized, Role: FOLLOWER
I20260809 03:37:41.094807 22705 consensus_queue.cc:260] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596dc42adfce45f1a6a8d1ff272b6917" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44979 } } peers { permanent_uuid: "22fa246ce82744b299ca9e6fc766287a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 43381 } } peers { permanent_uuid: "2376cbf128f34273848ac87f41d53942" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43259 } }
I20260809 03:37:41.096026 22706 log.cc:824] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:41.096098 22705 ts_tablet_manager.cc:1428] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:37:41.096514 22303 heartbeater.cc:503] Master 127.1.224.126:44207 was elected leader, sending a full tablet report...
I20260809 03:37:41.099252 22706 tablet_bootstrap.cc:492] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a: No bootstrap required, opened a new log
I20260809 03:37:41.099488 22706 ts_tablet_manager.cc:1397] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260809 03:37:41.103717 22706 raft_consensus.cc:348] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596dc42adfce45f1a6a8d1ff272b6917" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44979 } } peers { permanent_uuid: "22fa246ce82744b299ca9e6fc766287a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 43381 } } peers { permanent_uuid: "2376cbf128f34273848ac87f41d53942" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43259 } }
I20260809 03:37:41.104161 22706 raft_consensus.cc:374] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:41.104230 22706 raft_consensus.cc:729] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22fa246ce82744b299ca9e6fc766287a, State: Initialized, Role: FOLLOWER
I20260809 03:37:41.104861 22706 consensus_queue.cc:260] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596dc42adfce45f1a6a8d1ff272b6917" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44979 } } peers { permanent_uuid: "22fa246ce82744b299ca9e6fc766287a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 43381 } } peers { permanent_uuid: "2376cbf128f34273848ac87f41d53942" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43259 } }
I20260809 03:37:41.105676 22479 heartbeater.cc:503] Master 127.1.224.126:44207 was elected leader, sending a full tablet report...
I20260809 03:37:41.106232 22706 ts_tablet_manager.cc:1428] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260809 03:37:41.182158 22714 raft_consensus.cc:482] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:37:41.182554 22714 raft_consensus.cc:504] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596dc42adfce45f1a6a8d1ff272b6917" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44979 } } peers { permanent_uuid: "22fa246ce82744b299ca9e6fc766287a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 43381 } } peers { permanent_uuid: "2376cbf128f34273848ac87f41d53942" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43259 } }
W20260809 03:37:41.185182 22304 tablet.cc:2369] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:37:41.186817 22714 leader_election.cc:288] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 596dc42adfce45f1a6a8d1ff272b6917 (127.1.224.67:44979), 2376cbf128f34273848ac87f41d53942 (127.1.224.65:43259)
I20260809 03:37:41.189832 22700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.184661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:49729 (local address 127.1.224.67:44979)
0809 03:37:41.184920 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:41.184945 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:41.185026 (+ 81us) server_negotiation.cc:407] Connection header received
0809 03:37:41.185269 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:41.185272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:41.185328 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:41.185379 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:41.185872 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.186795 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.188435 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.189108 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.189208 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:41.189264 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:41.189610 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 03:37:41.189692 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260809 03:37:41.190447 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.184451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:44979 (local address 127.1.224.66:49729)
0809 03:37:41.184907 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:41.184924 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:41.185125 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:41.185553 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:41.185556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:41.185578 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:41.185732 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:41.185739 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.186946 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.186950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:41.188274 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:41.188288 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.189363 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.189367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:41.189466 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:41.189490 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:37:41.190214 (+ 724us) client_negotiation.cc:240] Negotiation successful
0809 03:37:41.190294 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":153,"threads_started":1}
I20260809 03:37:41.191397 22585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4962572e5ba424e9a9cf39f54ea5d56" candidate_uuid: "22fa246ce82744b299ca9e6fc766287a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "596dc42adfce45f1a6a8d1ff272b6917" is_pre_election: true
I20260809 03:37:41.191660 22702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.186414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:55693 (local address 127.1.224.65:43259)
0809 03:37:41.186530 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:41.186534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:41.187246 (+ 712us) server_negotiation.cc:407] Connection header received
0809 03:37:41.187434 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:41.187437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:41.187482 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:41.187528 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:41.187941 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.188776 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.190259 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.190873 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.190966 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:41.191023 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:41.191458 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 03:37:41.191542 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:37:41.191761 22585 raft_consensus.cc:2454] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22fa246ce82744b299ca9e6fc766287a in term 0.
I20260809 03:37:41.192029 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.186178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:43259 (local address 127.1.224.66:55693)
0809 03:37:41.187154 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:41.187169 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:41.187314 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:41.187659 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:41.187662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:41.187678 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:41.187806 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:41.187811 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.188924 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.188927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:41.190092 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:41.190105 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.191198 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.191202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:41.191302 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:41.191323 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:37:41.191803 (+ 480us) client_negotiation.cc:240] Negotiation successful
0809 03:37:41.191877 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":126,"threads_started":1}
I20260809 03:37:41.192703 22332 leader_election.cc:302] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22fa246ce82744b299ca9e6fc766287a, 596dc42adfce45f1a6a8d1ff272b6917; no voters:
I20260809 03:37:41.192924 22241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4962572e5ba424e9a9cf39f54ea5d56" candidate_uuid: "22fa246ce82744b299ca9e6fc766287a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2376cbf128f34273848ac87f41d53942" is_pre_election: true
I20260809 03:37:41.193235 22241 raft_consensus.cc:2454] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22fa246ce82744b299ca9e6fc766287a in term 0.
I20260809 03:37:41.193228 22714 raft_consensus.cc:2781] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:37:41.193320 22714 raft_consensus.cc:482] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:37:41.193367 22714 raft_consensus.cc:3037] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:41.194831 22714 raft_consensus.cc:504] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596dc42adfce45f1a6a8d1ff272b6917" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44979 } } peers { permanent_uuid: "22fa246ce82744b299ca9e6fc766287a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 43381 } } peers { permanent_uuid: "2376cbf128f34273848ac87f41d53942" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43259 } }
I20260809 03:37:41.195255 22714 leader_election.cc:288] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [CANDIDATE]: Term 1 election: Requested vote from peers 596dc42adfce45f1a6a8d1ff272b6917 (127.1.224.67:44979), 2376cbf128f34273848ac87f41d53942 (127.1.224.65:43259)
I20260809 03:37:41.195624 22585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4962572e5ba424e9a9cf39f54ea5d56" candidate_uuid: "22fa246ce82744b299ca9e6fc766287a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "596dc42adfce45f1a6a8d1ff272b6917"
I20260809 03:37:41.195758 22585 raft_consensus.cc:3037] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:41.195791 22241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4962572e5ba424e9a9cf39f54ea5d56" candidate_uuid: "22fa246ce82744b299ca9e6fc766287a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2376cbf128f34273848ac87f41d53942"
I20260809 03:37:41.195907 22241 raft_consensus.cc:3037] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:41.197297 22241 raft_consensus.cc:2454] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22fa246ce82744b299ca9e6fc766287a in term 1.
I20260809 03:37:41.197337 22585 raft_consensus.cc:2454] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22fa246ce82744b299ca9e6fc766287a in term 1.
I20260809 03:37:41.197693 22335 leader_election.cc:302] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22fa246ce82744b299ca9e6fc766287a, 2376cbf128f34273848ac87f41d53942; no voters:
I20260809 03:37:41.197930 22714 raft_consensus.cc:2781] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:37:41.198194 22714 raft_consensus.cc:686] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 1 LEADER]: Becoming Leader. State: Replica: 22fa246ce82744b299ca9e6fc766287a, State: Running, Role: LEADER
I20260809 03:37:41.198698 22714 consensus_queue.cc:237] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596dc42adfce45f1a6a8d1ff272b6917" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44979 } } peers { permanent_uuid: "22fa246ce82744b299ca9e6fc766287a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 43381 } } peers { permanent_uuid: "2376cbf128f34273848ac87f41d53942" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43259 } }
I20260809 03:37:41.202571 22016 catalog_manager.cc:5495] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a reported cstate change: term changed from 0 to 1, leader changed from <none> to 22fa246ce82744b299ca9e6fc766287a (127.1.224.66). New cstate: current_term: 1 leader_uuid: "22fa246ce82744b299ca9e6fc766287a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596dc42adfce45f1a6a8d1ff272b6917" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22fa246ce82744b299ca9e6fc766287a" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 43381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2376cbf128f34273848ac87f41d53942" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43259 } health_report { overall_health: UNKNOWN } } }
I20260809 03:37:41.217644 22699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.212255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35664 (local address 127.1.224.66:43381)
0809 03:37:41.212371 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:41.212375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:41.212389 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:37:41.212553 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:41.212556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:41.212633 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:41.212747 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:41.213166 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.213980 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.215288 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:41.215620 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:41.215713 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:41.216162 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:41.216808 (+ 646us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:41.216923 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:41.217400 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 03:37:41.217469 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 03:37:41.221433 22482 tablet.cc:2369] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:37:41.225275 22241 raft_consensus.cc:1261] T d4962572e5ba424e9a9cf39f54ea5d56 P 2376cbf128f34273848ac87f41d53942 [term 1 FOLLOWER]: Refusing update from remote peer 22fa246ce82744b299ca9e6fc766287a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:37:41.226074 22714 consensus_queue.cc:1035] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2376cbf128f34273848ac87f41d53942" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 43259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:37:41.234303 22585 raft_consensus.cc:1261] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917 [term 1 FOLLOWER]: Refusing update from remote peer 22fa246ce82744b299ca9e6fc766287a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:37:41.235165 22714 consensus_queue.cc:1035] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [LEADER]: Connected to new peer: Peer: permanent_uuid: "596dc42adfce45f1a6a8d1ff272b6917" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:37:41.237226 22675 tablet.cc:2369] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:37:41.242720 22724 mvcc.cc:205] Tried to move back new op lower bound from 7316466324370403328 to 7316466324272979968. Current Snapshot: MvccSnapshot[applied={T|T < 7316466324370403328}]
I20260809 03:37:41.244386 22726 mvcc.cc:205] Tried to move back new op lower bound from 7316466324370403328 to 7316466324272979968. Current Snapshot: MvccSnapshot[applied={T|T < 7316466324370403328}]
I20260809 03:37:41.259752 22017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:37:41.259977 22017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:37:41.263265 22017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5502cfbf62f5419c8eb381c3f50554ee: Sending DeleteTablet for 3 replicas of tablet d4962572e5ba424e9a9cf39f54ea5d56
I20260809 03:37:41.264122 22569 tablet_service.cc:1514] Processing DeleteTablet for tablet d4962572e5ba424e9a9cf39f54ea5d56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:37:41 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:38148
I20260809 03:37:41.264273 22391 tablet_service.cc:1514] Processing DeleteTablet for tablet d4962572e5ba424e9a9cf39f54ea5d56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:37:41 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:35652
I20260809 03:37:41.265376 22214 tablet_service.cc:1514] Processing DeleteTablet for tablet d4962572e5ba424e9a9cf39f54ea5d56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:37:41 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:36324
I20260809 03:37:41.266155 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 22071
I20260809 03:37:41.275969 22730 mvcc.cc:205] Tried to move back new op lower bound from 7316466324370403328 to 7316466324272979968. Current Snapshot: MvccSnapshot[applied={T|T < 7316466324370403328 or (T in {7316466324370403328})}]
I20260809 03:37:41.276820 22739 tablet_replica.cc:330] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917: stopping tablet replica
I20260809 03:37:41.277277 22739 raft_consensus.cc:2229] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:37:41.277748 22739 raft_consensus.cc:2258] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 03:37:41 dist-test-slave-jq4t krb5kdc[20593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for HTTP/127.1.224.84@KRBTEST.COM
I20260809 03:37:41.286217 22739 ts_tablet_manager.cc:1905] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:37:41.289510 22739 ts_tablet_manager.cc:1918] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:37:41.289906 22740 tablet_replica.cc:330] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a: stopping tablet replica
I20260809 03:37:41.290382 22740 raft_consensus.cc:2229] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:37:41.290982 22740 raft_consensus.cc:2258] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:37:41.298074 22740 ts_tablet_manager.cc:1905] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:37:41.300223 22739 log.cc:1196] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/d4962572e5ba424e9a9cf39f54ea5d56
I20260809 03:37:41.302196 22739 ts_tablet_manager.cc:1939] T d4962572e5ba424e9a9cf39f54ea5d56 P 596dc42adfce45f1a6a8d1ff272b6917: Deleting consensus metadata
I20260809 03:37:41.312839 21994 catalog_manager.cc:4841] TS 596dc42adfce45f1a6a8d1ff272b6917 (127.1.224.67:44979): tablet d4962572e5ba424e9a9cf39f54ea5d56 (table test-table [id=075261b4e2c64983ac2dab6641bdc5d2]) successfully deleted
I20260809 03:37:41.316919 22740 ts_tablet_manager.cc:1918] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:37:41.320336 22740 log.cc:1196] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/d4962572e5ba424e9a9cf39f54ea5d56
I20260809 03:37:41.325140 22740 ts_tablet_manager.cc:1939] T d4962572e5ba424e9a9cf39f54ea5d56 P 22fa246ce82744b299ca9e6fc766287a: Deleting consensus metadata
I20260809 03:37:41.326869 21993 catalog_manager.cc:4841] TS 22fa246ce82744b299ca9e6fc766287a (127.1.224.66:43381): tablet d4962572e5ba424e9a9cf39f54ea5d56 (table test-table [id=075261b4e2c64983ac2dab6641bdc5d2]) successfully deleted
W20260809 03:37:41.327451 21997 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:43259 (error 108)
I20260809 03:37:41.328637 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 22306
I20260809 03:37:41.357729 22701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:41.357341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:43259 (local address 127.0.0.1:36340)
0809 03:37:41.357506 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:41.357615 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:43259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 03:37:41.358158 21997 catalog_manager.cc:4584] TS 2376cbf128f34273848ac87f41d53942 (127.1.224.65:43259): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4962572e5ba424e9a9cf39f54ea5d56: Network error: Client connection negotiation failed: client connection to 127.1.224.65:43259: connect: Connection refused (error 111)
I20260809 03:37:41.375998 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 22483
I20260809 03:37:41.399403 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 21974
I20260809 03:37:41.415771 1921 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 03:37:41.771955 1921 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 03:37:41.772117 1921 mini_ranger.cc:66] Stopping Ranger...
I20260809 03:37:42.118961 1921 mini_ranger.cc:68] Stopped Ranger
2026-08-09 03:37:42.119 UTC [20645] LOG: received smart shutdown request
2026-08-09 03:37:42.123 UTC [20645] LOG: background worker "logical replication launcher" (PID 20660) exited with exit code 1
2026-08-09 03:37:42.123 UTC [20655] LOG: shutting down
2026-08-09 03:37:42.281 UTC [20645] LOG: database system is shut down
2026-08-09T03:37:42Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50038 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:37:42 dist-test-slave-jq4t krb5kdc[22754](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:37:42 dist-test-slave-jq4t krb5kdc[22754](info): set up 2 sockets
Aug 09 03:37:42 dist-test-slave-jq4t krb5kdc[22754](info): commencing operation
krb5kdc: starting...
W20260809 03:37:43.691761 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:37:43 dist-test-slave-jq4t krb5kdc[22754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:37:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:37:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:37:43.824359 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:33557
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:33557,127.1.224.125:44527,127.1.224.124:43893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:37:43.944068 22776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:43.944413 22776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:43.944510 22776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:43.948555 22776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:37:43.948668 22776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:43.948704 22776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:43.948745 22776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:37:43.948786 22776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:37:43.954245 22776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:33557,127.1.224.125:44527,127.1.224.124:43893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:33557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:43.955394 22776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:43.956537 22776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:43.965814 22781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:43.965816 22783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:43.965794 22786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:43.966178 22776 server_base.cc:1034] running on GCE node
I20260809 03:37:43.966843 22776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:43.968063 22776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:43.969267 22776 hybrid_clock.cc:648] HybridClock initialized: now 1786246663969241 us; error 43 us; skew 500 ppm
Aug 09 03:37:43 dist-test-slave-jq4t krb5kdc[22754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:43.972244 22776 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:37:43.973328 22776 webserver.cc:460] Webserver started at http://127.1.224.126:43429/ using document root <none> and password file <none>
I20260809 03:37:43.973941 22776 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:43.974035 22776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:43.974291 22776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:37:43.976468 22776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "3759101060f0464fb1ceb0c595689292"
format_stamp: "Formatted at 2026-08-09 03:37:43 on dist-test-slave-jq4t"
server_key: "4e013191139bd1bd6f291c5ffe632b58"
server_key_iv: "34bf846a238ca28cf73dcdd372fc6dc5"
server_key_version: "encryptionkey@0"
I20260809 03:37:43.977164 22776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "3759101060f0464fb1ceb0c595689292"
format_stamp: "Formatted at 2026-08-09 03:37:43 on dist-test-slave-jq4t"
server_key: "4e013191139bd1bd6f291c5ffe632b58"
server_key_iv: "34bf846a238ca28cf73dcdd372fc6dc5"
server_key_version: "encryptionkey@0"
I20260809 03:37:43.981225 22776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 03:37:43.983891 22795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:43.985301 22776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:37:43.985466 22776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "3759101060f0464fb1ceb0c595689292"
format_stamp: "Formatted at 2026-08-09 03:37:43 on dist-test-slave-jq4t"
server_key: "4e013191139bd1bd6f291c5ffe632b58"
server_key_iv: "34bf846a238ca28cf73dcdd372fc6dc5"
server_key_version: "encryptionkey@0"
I20260809 03:37:43.985603 22776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:44.006942 22776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:44.010782 22776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:44.011044 22776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:44.019251 22776 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:33557
I20260809 03:37:44.019276 22867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:33557 every 8 connection(s)
I20260809 03:37:44.020326 22776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:37:44.022496 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 22776
I20260809 03:37:44.022634 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:37:44.022960 1921 external_mini_cluster.cc:1428] Setting key 642b1bbb39b1fb9745033675d4490172
I20260809 03:37:44.023715 22868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:44.028192 22868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.224.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.125@KRBTEST.COM" created.
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:44.045996 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.030780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:33557 (local address 127.0.0.1:34718)
0809 03:37:44.031481 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.031541 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:44.032701 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:44.033466 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:44.033470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:44.033802 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:44.034173 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.034190 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.035669 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.035675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.036831 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.036843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.037523 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.037527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.037684 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.038343 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:44.038371 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:44.040179 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:44.042796 (+ 2617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.042805 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.042815 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.043062 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.043471 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.043473 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.043475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.043597 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.044433 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:44.044439 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:44.044657 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 03:37:44.044811 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 03:37:44.045093 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":1,"thread_start_us":171,"threads_started":1}
I20260809 03:37:44.045994 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.030780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34718 (local address 127.1.224.126:33557)
0809 03:37:44.031432 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.031450 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.031679 (+ 229us) server_negotiation.cc:407] Connection header received
0809 03:37:44.033064 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.033070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.033261 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.033426 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.034329 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.035525 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.036973 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.037357 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.037600 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.040361 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.040372 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.040389 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.040419 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.042664 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.043187 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.043197 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.043200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.043361 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.043691 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.043695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.043697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.044308 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.044506 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.044937 (+ 431us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.045192 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":520,"spinlock_wait_cycles":57088,"thread_start_us":177,"threads_started":1}
I20260809 03:37:44.048692 22868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:37:44.049458 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.049187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:44527 (local address 127.0.0.1:39880)
0809 03:37:44.049315 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.049395 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 03:37:44.049698 22798 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.224.125:44527: connect: Connection refused (error 111)
I20260809 03:37:44.050616 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.050402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:44527 (local address 127.0.0.1:39894)
0809 03:37:44.050513 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.050565 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 03:37:44.050743 22868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.125:44527: Network error: Client connection negotiation failed: client connection to 127.1.224.125:44527: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.125.keytab.
Entry for principal kudu/127.1.224.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.125.keytab.
I20260809 03:37:44.053571 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.125:44527
--webserver_interface=127.1.224.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.125.keytab
--principal=kudu/127.1.224.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:33557,127.1.224.125:44527,127.1.224.124:43893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:37:44.100014 22868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } attempt: 1
I20260809 03:37:44.101058 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.100721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:44527 (local address 127.0.0.1:39898)
0809 03:37:44.100889 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.100957 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 03:37:44.101853 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.101620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:44527 (local address 127.0.0.1:39908)
0809 03:37:44.101737 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.101791 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 03:37:44.101976 22868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.125:44527: Network error: Client connection negotiation failed: client connection to 127.1.224.125:44527: connect: Connection refused (error 111)
I20260809 03:37:44.170137 22868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } attempt: 2
W20260809 03:37:44.170449 22882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:44.170784 22882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:44.170873 22882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 03:37:44.171110 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.170743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:44527 (local address 127.0.0.1:39918)
0809 03:37:44.170948 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.171030 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 03:37:44.171866 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.171673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:44527 (local address 127.0.0.1:39926)
0809 03:37:44.171767 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.171812 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 03:37:44.172052 22868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.125:44527: Network error: Client connection negotiation failed: client connection to 127.1.224.125:44527: connect: Connection refused (error 111)
W20260809 03:37:44.175202 22882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:37:44.175279 22882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:44.175328 22882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:44.175379 22882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:37:44.175414 22882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:37:44.180768 22882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:33557,127.1.224.125:44527,127.1.224.124:43893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.125.keytab
--principal=kudu/127.1.224.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.125:44527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.224.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:44.181855 22882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:44.182924 22882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:44.191711 22891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:44.191746 22893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:44.191751 22890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:44.192103 22882 server_base.cc:1034] running on GCE node
I20260809 03:37:44.192765 22882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:44.193893 22882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:44.195065 22882 hybrid_clock.cc:648] HybridClock initialized: now 1786246664195048 us; error 29 us; skew 500 ppm
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246664, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:44.198011 22882 init.cc:377] Logged in from keytab as kudu/127.1.224.125@KRBTEST.COM (short username kudu)
I20260809 03:37:44.198926 22882 webserver.cc:460] Webserver started at http://127.1.224.125:34303/ using document root <none> and password file <none>
I20260809 03:37:44.199498 22882 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:44.199569 22882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:44.199752 22882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:37:44.201979 22882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/data/instance:
uuid: "7bdcc5183e6b4596bafaca5aac32b878"
format_stamp: "Formatted at 2026-08-09 03:37:44 on dist-test-slave-jq4t"
server_key: "0631e98f48d20323f23530f901888aac"
server_key_iv: "011b120256b8f2a86cf6a0752d369826"
server_key_version: "encryptionkey@0"
I20260809 03:37:44.202548 22882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/wal/instance:
uuid: "7bdcc5183e6b4596bafaca5aac32b878"
format_stamp: "Formatted at 2026-08-09 03:37:44 on dist-test-slave-jq4t"
server_key: "0631e98f48d20323f23530f901888aac"
server_key_iv: "011b120256b8f2a86cf6a0752d369826"
server_key_version: "encryptionkey@0"
I20260809 03:37:44.206607 22882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 03:37:44.209192 22903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:44.210477 22882 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:37:44.210642 22882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/wal
uuid: "7bdcc5183e6b4596bafaca5aac32b878"
format_stamp: "Formatted at 2026-08-09 03:37:44 on dist-test-slave-jq4t"
server_key: "0631e98f48d20323f23530f901888aac"
server_key_iv: "011b120256b8f2a86cf6a0752d369826"
server_key_version: "encryptionkey@0"
I20260809 03:37:44.210762 22882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:44.225498 22882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:44.228672 22882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:44.228896 22882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:44.237162 22882 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.125:44527
I20260809 03:37:44.237247 22976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.125:44527 every 8 connection(s)
I20260809 03:37:44.238268 22882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
I20260809 03:37:44.240768 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 22882
I20260809 03:37:44.240908 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-1/wal/instance
I20260809 03:37:44.241231 1921 external_mini_cluster.cc:1428] Setting key 2c1bc3a562f82909d81f1ad32ba2a086
I20260809 03:37:44.241703 22977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:44.246261 22977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.224.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.124@KRBTEST.COM" created.
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246664, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.125@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:44.261355 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.248409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.1.224.126:33557)
0809 03:37:44.248585 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.248589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.249751 (+ 1162us) server_negotiation.cc:407] Connection header received
0809 03:37:44.250755 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.250759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.250825 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.250936 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.252263 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.253014 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.254310 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.254571 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.254688 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.257590 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.257594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.257596 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.257622 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.259133 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.259676 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.259679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.259680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.259758 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.260109 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.260113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.260115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.260308 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.260424 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.261022 (+ 598us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.261155 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:37:44.262182 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.249009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:33557 (local address 127.0.0.1:34722)
0809 03:37:44.249590 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.249633 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:44.250511 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:44.251121 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:44.251128 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:44.251564 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:44.252111 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.252125 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.253207 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.253211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.254179 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.254190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.254746 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.254750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.254939 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.255661 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:44.255690 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:44.257416 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:44.259259 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.259266 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.259281 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.259561 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.259851 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.259854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.259856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.260010 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.260431 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:44.260441 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:44.260762 (+ 321us) client_negotiation.cc:769] Sending connection context
0809 03:37:44.260963 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 03:37:44.261300 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":184,"threads_started":1}
I20260809 03:37:44.263180 22868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } attempt: 3
I20260809 03:37:44.263734 22977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246664, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.125@KRBTEST.COM for kudu/127.1.224.125@KRBTEST.COM
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for kudu/127.1.224.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.124.keytab.
Entry for principal kudu/127.1.224.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.124.keytab.
I20260809 03:37:44.271735 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.124:43893
--webserver_interface=127.1.224.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.124.keytab
--principal=kudu/127.1.224.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:33557,127.1.224.125:44527,127.1.224.124:43893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:37:44.275343 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.263607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:44527 (local address 127.0.0.1:39936)
0809 03:37:44.263827 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.263843 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:44.263927 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:44.264801 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:44.264805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:44.264991 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:44.265610 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.265621 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.266657 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.266660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.267342 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.267353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.267961 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.267965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.268074 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.268768 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:44.268780 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:44.270401 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:44.273705 (+ 3304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.273708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.273710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.273915 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.274240 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.274243 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.274244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.274317 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.274733 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:44.274736 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:44.274842 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 03:37:44.274967 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 03:37:44.275125 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 03:37:44.275449 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.264246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:44527 (local address 127.0.0.1:39950)
0809 03:37:44.264407 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.264420 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:44.264523 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:44.264962 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:44.264964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:44.265151 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:44.265242 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.265248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.266467 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.266471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.267125 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.267134 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.267913 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.267916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.268010 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.268550 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:44.268566 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:44.270416 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:44.273354 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.273360 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.273363 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.273641 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.274010 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.274015 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.274017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.274118 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.274815 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:44.274817 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:44.274915 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 03:37:44.275030 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 03:37:44.275194 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":30}
I20260809 03:37:44.276144 22985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.264344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.1.224.125:44527)
0809 03:37:44.264694 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.264701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.264713 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:37:44.264762 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.264765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.264819 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.264917 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.265363 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.266360 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.267244 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.267787 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.267918 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.270608 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.270611 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.270613 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.270644 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.273230 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.273766 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.273775 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.273778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.273890 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.274219 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.274226 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.274229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.274647 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.274771 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.275702 (+ 931us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.275880 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":279,"spinlock_wait_cycles":5376,"thread_start_us":142,"threads_started":1}
I20260809 03:37:44.277499 22984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.263720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.1.224.125:44527)
0809 03:37:44.264041 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.264050 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.264089 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:37:44.264190 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.264197 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.264680 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.264780 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.265771 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.266552 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.267450 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.267810 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.267959 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.270578 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.270590 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.270605 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.270644 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.273575 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.274051 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.274057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.274062 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.274128 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.274422 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.274424 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.274425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.274623 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.274771 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.277206 (+ 2435us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.277326 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":224,"spinlock_wait_cycles":10624,"thread_start_us":105,"threads_started":1}
I20260809 03:37:44.278568 22977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:37:44.278604 22868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:37:44.279363 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.279114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37364)
0809 03:37:44.279244 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.279302 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 03:37:44.279413 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.279057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37354)
0809 03:37:44.279220 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.279328 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 03:37:44.279819 22905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
I20260809 03:37:44.280543 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.279909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37372)
0809 03:37:44.280329 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.280405 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":133,"threads_started":1}
W20260809 03:37:44.280750 22868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:43893: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
I20260809 03:37:44.281057 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.280867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37384)
0809 03:37:44.280964 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.281009 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 03:37:44.281270 22977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:43893: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
I20260809 03:37:44.311942 22868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } attempt: 1
I20260809 03:37:44.312968 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.312628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37390)
0809 03:37:44.312787 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.312880 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 03:37:44.313715 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.313496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37392)
0809 03:37:44.313608 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.313658 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 03:37:44.313859 22868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:43893: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
I20260809 03:37:44.330549 22977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } attempt: 1
I20260809 03:37:44.331575 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.331234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37406)
0809 03:37:44.331418 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.331487 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 03:37:44.332373 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.332136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37422)
0809 03:37:44.332250 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.332303 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 03:37:44.332527 22977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:43893: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
I20260809 03:37:44.389036 22868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } attempt: 2
I20260809 03:37:44.389986 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.389634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37434)
0809 03:37:44.389804 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.389901 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 03:37:44.390743 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.390529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37444)
0809 03:37:44.390638 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.390688 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 03:37:44.390882 22868 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:43893: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
W20260809 03:37:44.397615 22986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:44.397938 22986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:44.398054 22986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 03:37:44.400691 22977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } attempt: 2
I20260809 03:37:44.401531 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.401231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37446)
0809 03:37:44.401385 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.401451 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 03:37:44.402281 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.402067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37450)
0809 03:37:44.402176 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.402224 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 03:37:44.402422 22977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:43893: Network error: Client connection negotiation failed: client connection to 127.1.224.124:43893: connect: Connection refused (error 111)
W20260809 03:37:44.402595 22986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:37:44.402670 22986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:44.402714 22986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:44.402752 22986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:37:44.402788 22986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:37:44.408345 22986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:33557,127.1.224.125:44527,127.1.224.124:43893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.124.keytab
--principal=kudu/127.1.224.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.124:43893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.224.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:44.409708 22986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:44.410811 22986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:44.420198 22997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:44.420425 22986 server_base.cc:1034] running on GCE node
W20260809 03:37:44.420198 22995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:44.420198 22999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:44.421258 22986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:44.422487 22986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:44.423691 22986 hybrid_clock.cc:648] HybridClock initialized: now 1786246664423673 us; error 29 us; skew 500 ppm
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246664, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:44.426744 22986 init.cc:377] Logged in from keytab as kudu/127.1.224.124@KRBTEST.COM (short username kudu)
I20260809 03:37:44.427721 22986 webserver.cc:460] Webserver started at http://127.1.224.124:38221/ using document root <none> and password file <none>
I20260809 03:37:44.428332 22986 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:44.428427 22986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:44.428696 22986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:37:44.430928 22986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/data/instance:
uuid: "4b5e2acf037c44738d11783d60e90fea"
format_stamp: "Formatted at 2026-08-09 03:37:44 on dist-test-slave-jq4t"
server_key: "98f3c8e1b5004c945be14b46a352cf6d"
server_key_iv: "95e1b56a9c04285cd3680065201bf84b"
server_key_version: "encryptionkey@0"
I20260809 03:37:44.431555 22986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/wal/instance:
uuid: "4b5e2acf037c44738d11783d60e90fea"
format_stamp: "Formatted at 2026-08-09 03:37:44 on dist-test-slave-jq4t"
server_key: "98f3c8e1b5004c945be14b46a352cf6d"
server_key_iv: "95e1b56a9c04285cd3680065201bf84b"
server_key_version: "encryptionkey@0"
I20260809 03:37:44.435721 22986 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:37:44.438741 23007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:44.440131 22986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:37:44.440260 22986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/wal
uuid: "4b5e2acf037c44738d11783d60e90fea"
format_stamp: "Formatted at 2026-08-09 03:37:44 on dist-test-slave-jq4t"
server_key: "98f3c8e1b5004c945be14b46a352cf6d"
server_key_iv: "95e1b56a9c04285cd3680065201bf84b"
server_key_version: "encryptionkey@0"
I20260809 03:37:44.440369 22986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:44.454366 22986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:44.457940 22986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:44.458113 22986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:44.465703 22986 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.124:43893
I20260809 03:37:44.465736 23080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.124:43893 every 8 connection(s)
I20260809 03:37:44.466853 22986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
I20260809 03:37:44.470026 23081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:44.470688 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 22986
I20260809 03:37:44.470818 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786246494448707-1921-0/minicluster-data/master-2/wal/instance
I20260809 03:37:44.471171 1921 external_mini_cluster.cc:1428] Setting key b2d9e2cb9f2a66be71cb616c8978e547
I20260809 03:37:44.475468 23081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.124@KRBTEST.COM
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246664, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.124@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:44.485920 23084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.472681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37464 (local address 127.1.224.124:43893)
0809 03:37:44.473130 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.473146 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.473185 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:37:44.473890 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.473908 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.474242 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.474529 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.475498 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.476655 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.477679 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.478409 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.478589 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.481287 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.481294 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.481307 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.481341 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.483071 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.483512 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.483518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.483529 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.483635 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.483941 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.483944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.483946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.484349 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.484612 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.484862 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.485152 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":149,"threads_started":1}
I20260809 03:37:44.486865 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.477321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:33557 (local address 127.0.0.1:34734)
0809 03:37:44.477731 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.477780 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:44.477909 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:44.478274 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:44.478279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:44.478613 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:44.478744 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.478750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.479796 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.479798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.480567 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.480578 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.481140 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.481144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.481256 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.481790 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:44.481810 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:44.483415 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:44.485187 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.485194 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.485198 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.485417 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.485705 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.485710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.485712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.485789 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.486247 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:44.486255 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:44.486370 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 03:37:44.486508 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 03:37:44.486685 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":149,"threads_started":1}
I20260809 03:37:44.486846 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.477445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.1.224.126:33557)
0809 03:37:44.477597 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.477603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.477867 (+ 264us) server_negotiation.cc:407] Connection header received
0809 03:37:44.478019 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.478023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.478134 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.478216 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.478876 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.479656 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.480683 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.480983 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.481098 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.483543 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.483547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.483554 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.483580 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.485050 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.485522 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.485526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.485527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.485614 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.485886 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.485889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.485891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.486137 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.486247 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.486570 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.486685 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:37:44.488371 23081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:37:44.490053 22868 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } attempt: 3
I20260809 03:37:44.493583 22977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } attempt: 3
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246664, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.124@KRBTEST.COM for kudu/127.1.224.125@KRBTEST.COM
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for kudu/127.1.224.124@KRBTEST.COM
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:44.499585 22984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.489004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39958 (local address 127.1.224.125:44527)
0809 03:37:44.489142 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.489146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.489157 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:37:44.489237 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.489240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.489286 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.489374 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.490390 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.491168 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.492023 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.492317 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.492435 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.495043 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.495048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.495051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.495080 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.497214 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.497835 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.497840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.497843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.497960 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.498325 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.498331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.498336 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.498611 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.498699 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.499172 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.499342 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 03:37:44.500155 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.488898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:44527 (local address 127.0.0.1:39958)
0809 03:37:44.489087 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.489097 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:44.489154 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:44.489409 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:44.489412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:44.489597 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:44.490238 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.490252 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.491325 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.491328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.491897 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.491906 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.492441 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.492445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.492584 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.493186 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:44.493200 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:44.494844 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:44.497340 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.497346 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.497353 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.497712 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.498093 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.498097 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.498102 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.498204 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.498786 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:44.498790 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:44.498958 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 03:37:44.499811 (+ 853us) client_negotiation.cc:240] Negotiation successful
0809 03:37:44.499962 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":27}
I20260809 03:37:44.500965 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.488224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34740 (local address 127.1.224.126:33557)
0809 03:37:44.488333 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.488336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.488846 (+ 510us) server_negotiation.cc:407] Connection header received
0809 03:37:44.489221 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.489225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.489272 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.489394 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.490307 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.491103 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.492167 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.493287 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.493367 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.496834 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.496838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.496841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.496879 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.498543 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.499062 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.499068 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.499073 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.499159 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.499495 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.499499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.499501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.499736 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.500566 (+ 830us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.500655 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.500794 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":774,"server-negotiator.queue_time_us":44,"spinlock_wait_cycles":2432}
I20260809 03:37:44.500926 23081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246664, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.125@KRBTEST.COM for kudu/127.1.224.124@KRBTEST.COM
I20260809 03:37:44.502028 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.490438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37478)
0809 03:37:44.490547 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.490562 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:44.490650 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:44.490960 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:44.490964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:44.491162 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:44.491304 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.491313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.492280 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.492285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.493122 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.493134 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.493769 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.493773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.493896 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.494364 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:44.494384 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:44.496864 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:44.499680 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.499684 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.499685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.499949 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.500266 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.500269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.500271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.500370 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.501481 (+ 1111us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:44.501484 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:44.501588 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 03:37:44.501677 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 03:37:44.501832 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 03:37:44.502583 23084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.490627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37478 (local address 127.1.224.124:43893)
0809 03:37:44.490745 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.490749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.490762 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:37:44.490794 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.490796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.490844 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.490923 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.491423 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.492170 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.493243 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.493608 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.494201 (+ 593us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.497033 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.497038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.497040 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.497067 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.499303 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.500074 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.500077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.500079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.500154 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.500713 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.500719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.500724 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.501344 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.502213 (+ 869us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.502294 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.502431 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:37:44.506904 22981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.494033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37486)
0809 03:37:44.494228 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.494243 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:44.494316 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:44.496119 (+ 1803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:44.496122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:44.496265 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:44.496390 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.496396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.497513 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.497517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.498349 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.498377 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.499107 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.499114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.499214 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.499911 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:44.499926 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:44.502350 (+ 2424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:44.505210 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.505214 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.505216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.505496 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.505785 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.505788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.505790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.505887 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.506287 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:44.506290 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:44.506396 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 03:37:44.506515 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 03:37:44.506662 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":22}
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246664, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.124@KRBTEST.COM for kudu/127.1.224.124@KRBTEST.COM
I20260809 03:37:44.507638 23086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.494514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37486 (local address 127.1.224.124:43893)
0809 03:37:44.495880 (+ 1366us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.495890 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.495906 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:37:44.495945 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.495949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.495998 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.496094 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.496529 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.497360 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.498528 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.498977 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.499470 (+ 493us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.502634 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.502638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.502641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.503268 (+ 627us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.505088 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.505607 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.505611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.505613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.505691 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.505999 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.506002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.506004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.506185 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.506296 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.507281 (+ 985us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.507406 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":1296,"thread_start_us":120,"threads_started":1}
I20260809 03:37:44.507695 22868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292: Bootstrap starting.
Aug 09 03:37:44 dist-test-slave-jq4t krb5kdc[22754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.125@KRBTEST.COM
I20260809 03:37:44.511518 22977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878: Bootstrap starting.
I20260809 03:37:44.512183 23084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.501741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.1.224.124:43893)
0809 03:37:44.502750 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.502754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.502766 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:37:44.502797 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.502800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.502838 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.502911 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.503338 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.504068 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.505114 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.505395 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.505485 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.508345 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.508349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.508351 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.508373 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.510281 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.510759 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.510762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.510764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.510855 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.511122 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.511124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.511126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.511337 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.511413 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.511818 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.511956 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":952,"thread_start_us":123,"threads_started":1}
I20260809 03:37:44.512214 22868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:44.512904 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.501453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:43893 (local address 127.0.0.1:37500)
0809 03:37:44.501743 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:44.501757 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:44.501828 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:44.502949 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:44.502952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:44.503122 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:44.503197 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.503202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.504200 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.504204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.504990 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:44.505002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.505623 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.505626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:44.505696 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.506311 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:44.506328 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:44.508216 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:44.510404 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.510407 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.510410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.510655 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.510959 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:44.510961 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:44.510963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:44.511039 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:44.511472 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:44.511475 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:44.511560 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 03:37:44.512616 (+ 1056us) client_negotiation.cc:240] Negotiation successful
0809 03:37:44.512762 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
I20260809 03:37:44.513175 22984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.502412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39968 (local address 127.1.224.125:44527)
0809 03:37:44.502542 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.502549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.502565 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:37:44.502601 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.502604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.502644 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.502719 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.503605 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.504344 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.505491 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.505831 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.505959 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.508936 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.508940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.508942 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.508968 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.510646 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.511182 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.511185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.511187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.511272 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.511604 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.511609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.511610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.511819 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.511995 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.512728 (+ 733us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.512859 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:37:44.513423 22868 log.cc:824] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:44.514748 22977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:44.515540 22977 log.cc:824] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:44.516772 23081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea: Bootstrap starting.
I20260809 03:37:44.518030 22977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878: No bootstrap required, opened a new log
I20260809 03:37:44.518285 22868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292: No bootstrap required, opened a new log
I20260809 03:37:44.519157 23081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:44.519877 23081 log.cc:824] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:44.522511 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:37:44.522369 22977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } }
I20260809 03:37:44.522369 22868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } }
I20260809 03:37:44.522619 22977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:44.522619 22868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:44.522682 22868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3759101060f0464fb1ceb0c595689292, State: Initialized, Role: FOLLOWER
I20260809 03:37:44.522693 22977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bdcc5183e6b4596bafaca5aac32b878, State: Initialized, Role: FOLLOWER
I20260809 03:37:44.523196 22977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } }
I20260809 03:37:44.523195 22868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } }
I20260809 03:37:44.524586 23081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea: No bootstrap required, opened a new log
I20260809 03:37:44.524678 22868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:37:44.524678 22977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:37:44.528595 23092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } } }
I20260809 03:37:44.528749 23092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:44.529100 23081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } }
I20260809 03:37:44.529403 23081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:44.529470 23081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b5e2acf037c44738d11783d60e90fea, State: Initialized, Role: FOLLOWER
I20260809 03:37:44.530002 23081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } }
I20260809 03:37:44.531539 23081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [sys.catalog]: configured and running, proceeding with master startup.
W20260809 03:37:44.531922 23123 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:37:44.532021 23123 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:37:44.532895 23135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } } }
I20260809 03:37:44.533138 23135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:44.534179 23093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } } }
I20260809 03:37:44.534317 23093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 03:37:44.537715 23151 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:37:44.537892 23151 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:37:44.534543 22882 master_runner.cc:186] Error getting master registration for 127.1.224.124:43893: OK, instance_id { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" instance_seqno: 1786246664463568 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:37:44.538399 22882 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:37:44.539680 23092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:37:44.540729 23092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } }
I20260809 03:37:44.542001 22776 master_runner.cc:263] No leader master found from master 3759101060f0464fb1ceb0c595689292
I20260809 03:37:44.542094 22776 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 03:37:44.543618 23133 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:37:44.543736 23133 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:37:44.546428 22939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3759101060f0464fb1ceb0c595689292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bdcc5183e6b4596bafaca5aac32b878" is_pre_election: true
I20260809 03:37:44.546818 22939 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3759101060f0464fb1ceb0c595689292 in term 0.
I20260809 03:37:44.547302 22798 leader_election.cc:302] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3759101060f0464fb1ceb0c595689292, 7bdcc5183e6b4596bafaca5aac32b878; no voters:
I20260809 03:37:44.547890 23092 leader_election.cc:288] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bdcc5183e6b4596bafaca5aac32b878 (127.1.224.125:44527), 4b5e2acf037c44738d11783d60e90fea (127.1.224.124:43893)
I20260809 03:37:44.548010 23092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:37:44.548113 23092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:37:44.548180 23092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:44.548595 23045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3759101060f0464fb1ceb0c595689292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b5e2acf037c44738d11783d60e90fea" is_pre_election: true
I20260809 03:37:44.548918 23045 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3759101060f0464fb1ceb0c595689292 in term 0.
I20260809 03:37:44.549682 22986 master_runner.cc:263] No leader master found from master 4b5e2acf037c44738d11783d60e90fea
I20260809 03:37:44.549755 22986 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:37:44.549804 23092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } }
I20260809 03:37:44.550436 23084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.524960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.1.224.124:43893)
0809 03:37:44.525103 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.525108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.532612 (+ 7504us) server_negotiation.cc:407] Connection header received
0809 03:37:44.532765 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.532768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.532826 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.532884 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.534021 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.534801 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.536247 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.536649 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.536736 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.545109 (+ 8373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.545114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.545117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.545149 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.547487 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.548513 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.548518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.548520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.548629 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.549107 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.549110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.549112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.549373 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.549485 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.550107 (+ 622us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.550277 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:37:44.550493 22939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3759101060f0464fb1ceb0c595689292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bdcc5183e6b4596bafaca5aac32b878"
I20260809 03:37:44.550624 22939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:44.552634 22939 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3759101060f0464fb1ceb0c595689292 in term 1.
I20260809 03:37:44.552979 22798 leader_election.cc:302] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3759101060f0464fb1ceb0c595689292, 7bdcc5183e6b4596bafaca5aac32b878; no voters:
I20260809 03:37:44.553566 23045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3759101060f0464fb1ceb0c595689292" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b5e2acf037c44738d11783d60e90fea"
I20260809 03:37:44.553716 23045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:44.555078 23045 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3759101060f0464fb1ceb0c595689292 in term 1.
I20260809 03:37:44.555452 23092 leader_election.cc:288] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [CANDIDATE]: Term 1 election: Requested vote from peers 7bdcc5183e6b4596bafaca5aac32b878 (127.1.224.125:44527), 4b5e2acf037c44738d11783d60e90fea (127.1.224.124:43893)
I20260809 03:37:44.555631 23092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:37:44.555732 23092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [term 1 LEADER]: Becoming Leader. State: Replica: 3759101060f0464fb1ceb0c595689292, State: Running, Role: LEADER
I20260809 03:37:44.556123 23092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } }
I20260809 03:37:44.562523 22984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.526024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.1.224.125:44527)
0809 03:37:44.526130 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.526134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.552564 (+ 26430us) server_negotiation.cc:407] Connection header received
0809 03:37:44.552854 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.552857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.552916 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.552970 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.554130 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.554906 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.556254 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.556694 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.556783 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.558685 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.558689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.558692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.558721 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.560446 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.561004 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.561008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.561010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.561108 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.561463 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.561468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.561471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.561735 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.561849 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.562218 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.562360 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:37:44.565979 23154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3759101060f0464fb1ceb0c595689292. Latest consensus state: current_term: 1 leader_uuid: "3759101060f0464fb1ceb0c595689292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } } }
I20260809 03:37:44.566138 23154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:44.572911 23156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:37:44.574630 23156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:37:44.578017 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:44.525369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34748 (local address 127.1.224.126:33557)
0809 03:37:44.525495 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:44.525499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:44.544903 (+ 19404us) server_negotiation.cc:407] Connection header received
0809 03:37:44.560830 (+ 15927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:44.560835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:44.560896 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:44.560958 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:44.562840 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.563530 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.566867 (+ 3337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:44.567203 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:44.567289 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:44.573749 (+ 6460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:44.573756 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:44.573760 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:44.573793 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:44.575587 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.576137 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.576141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.576143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.576233 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:44.576590 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:44.576594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:44.576596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:44.576850 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:44.576970 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:44.577728 (+ 758us) server_negotiation.cc:299] Negotiation successful
0809 03:37:44.577865 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:37:44.582610 23045 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [term 1 FOLLOWER]: Refusing update from remote peer 3759101060f0464fb1ceb0c595689292: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:37:44.583204 22939 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [term 1 FOLLOWER]: Refusing update from remote peer 3759101060f0464fb1ceb0c595689292: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:37:44.583671 23092 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:37:44.583837 23154 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:37:44.588194 23093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3759101060f0464fb1ceb0c595689292. Latest consensus state: current_term: 1 leader_uuid: "3759101060f0464fb1ceb0c595689292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } } }
I20260809 03:37:44.588343 23093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:44.590404 23135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3759101060f0464fb1ceb0c595689292. Latest consensus state: current_term: 1 leader_uuid: "3759101060f0464fb1ceb0c595689292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } } }
I20260809 03:37:44.590567 23135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:44.596997 23154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3759101060f0464fb1ceb0c595689292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } } }
I20260809 03:37:44.597167 23154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:44.598373 23093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3759101060f0464fb1ceb0c595689292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } } }
I20260809 03:37:44.598507 23093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:44.598914 23157 mvcc.cc:205] Tried to move back new op lower bound from 7316466338120523776 to 7316466338047688704. Current Snapshot: MvccSnapshot[applied={T|T < 7316466338120523776}]
I20260809 03:37:44.599931 23159 mvcc.cc:205] Tried to move back new op lower bound from 7316466338120523776 to 7316466338047688704. Current Snapshot: MvccSnapshot[applied={T|T < 7316466338120523776}]
I20260809 03:37:44.606446 23156 catalog_manager.cc:1347] Generated new cluster ID: d13ad36f88644569b04b58c3c224eafa
I20260809 03:37:44.606530 23156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:37:44.606887 23092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3759101060f0464fb1ceb0c595689292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } } }
I20260809 03:37:44.607026 23092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:44.608589 23135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3759101060f0464fb1ceb0c595689292" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3759101060f0464fb1ceb0c595689292" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33557 } } peers { permanent_uuid: "7bdcc5183e6b4596bafaca5aac32b878" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 44527 } } peers { permanent_uuid: "4b5e2acf037c44738d11783d60e90fea" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 43893 } } }
I20260809 03:37:44.608727 23135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:37:44.608820 23161 mvcc.cc:205] Tried to move back new op lower bound from 7316466338120523776 to 7316466338047688704. Current Snapshot: MvccSnapshot[applied={T|T < 7316466338120523776}]
I20260809 03:37:44.627921 23156 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:37:44.629381 23156 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:37:44.645025 23156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3759101060f0464fb1ceb0c595689292: Generated new TSK 0
I20260809 03:37:44.646083 23156 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:37:45.539430 23151 catalog_manager.cc:1259] Loaded cluster ID: d13ad36f88644569b04b58c3c224eafa
I20260809 03:37:45.539533 23151 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea: loading cluster ID for follower catalog manager: success
I20260809 03:37:45.541369 23151 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea: acquiring CA information for follower catalog manager: success
I20260809 03:37:45.542637 23151 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4b5e2acf037c44738d11783d60e90fea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:37:45.545117 23133 catalog_manager.cc:1259] Loaded cluster ID: d13ad36f88644569b04b58c3c224eafa
I20260809 03:37:45.545185 23133 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878: loading cluster ID for follower catalog manager: success
I20260809 03:37:45.546787 23133 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878: acquiring CA information for follower catalog manager: success
I20260809 03:37:45.547969 23133 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7bdcc5183e6b4596bafaca5aac32b878: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:37:45.714936 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 22776
I20260809 03:37:45.729004 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 22882
I20260809 03:37:45.743947 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 22986
2026-08-09T03:37:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3337 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:37:45 dist-test-slave-jq4t krb5kdc[23175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:37:45 dist-test-slave-jq4t krb5kdc[23175](info): set up 2 sockets
Aug 09 03:37:45 dist-test-slave-jq4t krb5kdc[23175](info): commencing operation
krb5kdc: starting...
W20260809 03:37:47.024662 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:37:47 dist-test-slave-jq4t krb5kdc[23175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:37:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:37:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:37:47.164631 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:42557
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:42557
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:37:47.284679 23198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:47.285425 23198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:47.285558 23198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:47.289991 23198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:37:47.290077 23198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:47.290100 23198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:47.290117 23198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:37:47.290135 23198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:37:47.295933 23198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:42557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:42557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:47.297216 23198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:47.298275 23198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:47.307597 23206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:47.307518 23205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:47.307590 23198 server_base.cc:1034] running on GCE node
W20260809 03:37:47.307512 23208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:47.308601 23198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:47.309758 23198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:47.310968 23198 hybrid_clock.cc:648] HybridClock initialized: now 1786246667310949 us; error 33 us; skew 500 ppm
Aug 09 03:37:47 dist-test-slave-jq4t krb5kdc[23175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246667, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:47.314324 23198 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:37:47.315412 23198 webserver.cc:460] Webserver started at http://127.1.224.126:44143/ using document root <none> and password file <none>
I20260809 03:37:47.316059 23198 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:47.316162 23198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:47.316425 23198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:37:47.318751 23198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "e3b7fb71f1994a0f9b46b13f4218e9aa"
format_stamp: "Formatted at 2026-08-09 03:37:47 on dist-test-slave-jq4t"
server_key: "7389816fcb34cd0950670b947a5ade77"
server_key_iv: "489808d901c537ca9d49685f2575cad8"
server_key_version: "encryptionkey@0"
I20260809 03:37:47.319375 23198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "e3b7fb71f1994a0f9b46b13f4218e9aa"
format_stamp: "Formatted at 2026-08-09 03:37:47 on dist-test-slave-jq4t"
server_key: "7389816fcb34cd0950670b947a5ade77"
server_key_iv: "489808d901c537ca9d49685f2575cad8"
server_key_version: "encryptionkey@0"
I20260809 03:37:47.323460 23198 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 03:37:47.326254 23219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:47.327623 23198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 03:37:47.327786 23198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "e3b7fb71f1994a0f9b46b13f4218e9aa"
format_stamp: "Formatted at 2026-08-09 03:37:47 on dist-test-slave-jq4t"
server_key: "7389816fcb34cd0950670b947a5ade77"
server_key_iv: "489808d901c537ca9d49685f2575cad8"
server_key_version: "encryptionkey@0"
I20260809 03:37:47.327920 23198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:47.340830 23198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:47.344528 23198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:47.344769 23198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:47.352711 23198 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:42557
I20260809 03:37:47.352917 23292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:42557 every 8 connection(s)
I20260809 03:37:47.354095 23198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:37:47.357672 23293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:47.363111 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 23198
I20260809 03:37:47.363248 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:37:47.363554 1921 external_mini_cluster.cc:1428] Setting key 59a3ab45e11ee7237a4d21be5070f45d
I20260809 03:37:47.364131 23293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa: Bootstrap starting.
I20260809 03:37:47.366887 23293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:47.367729 23293 log.cc:824] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:47.370101 23293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa: No bootstrap required, opened a new log
Aug 09 03:37:47 dist-test-slave-jq4t krb5kdc[23175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:47.374845 23293 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b7fb71f1994a0f9b46b13f4218e9aa" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42557 } }
I20260809 03:37:47.375222 23293 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:47.375301 23293 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3b7fb71f1994a0f9b46b13f4218e9aa, State: Initialized, Role: FOLLOWER
I20260809 03:37:47.375864 23293 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b7fb71f1994a0f9b46b13f4218e9aa" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42557 } }
I20260809 03:37:47.376057 23293 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:37:47.376137 23293 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:37:47.376248 23293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:47.377648 23293 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b7fb71f1994a0f9b46b13f4218e9aa" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42557 } }
I20260809 03:37:47.378063 23293 leader_election.cc:302] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3b7fb71f1994a0f9b46b13f4218e9aa; no voters:
I20260809 03:37:47.378479 23293 leader_election.cc:288] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:37:47.378719 23300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:37:47.379041 23300 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [term 1 LEADER]: Becoming Leader. State: Replica: e3b7fb71f1994a0f9b46b13f4218e9aa, State: Running, Role: LEADER
I20260809 03:37:47.379492 23300 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b7fb71f1994a0f9b46b13f4218e9aa" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42557 } }
I20260809 03:37:47.379714 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:47.365127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48664 (local address 127.1.224.126:42557)
0809 03:37:47.365625 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:47.365642 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:47.365681 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:37:47.366369 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:47.366388 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:47.366747 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:47.367110 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:47.368025 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:47.369461 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:47.370465 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:47.371194 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:47.371375 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:47.373983 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:47.373990 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:47.374012 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:47.374048 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:47.376165 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:47.376733 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:47.376739 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:47.376745 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:47.376895 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:47.377183 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:47.377187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:47.377188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:47.377830 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:47.378043 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:47.378301 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 03:37:47.378655 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":167,"threads_started":1}
I20260809 03:37:47.379920 23293 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:37:47.381034 23302 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3b7fb71f1994a0f9b46b13f4218e9aa. Latest consensus state: current_term: 1 leader_uuid: "e3b7fb71f1994a0f9b46b13f4218e9aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b7fb71f1994a0f9b46b13f4218e9aa" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42557 } } }
I20260809 03:37:47.381168 23302 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:47.381778 23301 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3b7fb71f1994a0f9b46b13f4218e9aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b7fb71f1994a0f9b46b13f4218e9aa" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 42557 } } }
I20260809 03:37:47.382066 23301 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:47.382296 23311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:37:47.385826 23311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:37:47.387718 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:37:47.392292 23311 catalog_manager.cc:1347] Generated new cluster ID: cc8296a02686443181bf87ee15664fe5
I20260809 03:37:47.392385 23311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:37:47.413137 23311 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:37:47.414248 23311 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:37:47.418900 23311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3b7fb71f1994a0f9b46b13f4218e9aa: Generated new TSK 0
I20260809 03:37:47.419698 23311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:37:47.444865 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 23198
2026-08-09T03:37:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1704 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:37:47 dist-test-slave-jq4t krb5kdc[23327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:37:47 dist-test-slave-jq4t krb5kdc[23327](info): set up 2 sockets
Aug 09 03:37:47 dist-test-slave-jq4t krb5kdc[23327](info): commencing operation
krb5kdc: starting...
W20260809 03:37:48.721729 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:37:48 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:37:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:37:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:37:48.856319 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37957
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:37957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:37:48.976012 23346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:48.976361 23346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:48.976438 23346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:48.980516 23346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:37:48.980640 23346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:48.980674 23346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:48.980715 23346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:37:48.980748 23346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:37:48.986311 23346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:37957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:48.987612 23346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:48.988799 23346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:48.998047 23358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:48.998049 23355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:48.998183 23346 server_base.cc:1034] running on GCE node
W20260809 03:37:48.998049 23354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:48.999030 23346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:49.000267 23346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:49.001462 23346 hybrid_clock.cc:648] HybridClock initialized: now 1786246669001419 us; error 57 us; skew 500 ppm
Aug 09 03:37:48 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246668, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:49.004447 23346 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:37:49.005430 23346 webserver.cc:460] Webserver started at http://127.1.224.126:39549/ using document root <none> and password file <none>
I20260809 03:37:49.006006 23346 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:49.006065 23346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:49.006238 23346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:37:49.008440 23346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "5ae42346c98041c6a0ed665b75da1847"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "9406594f6267e54448731ab092975f62"
server_key_iv: "4a7a5de7c70b8cd4c6d889c4e03140a8"
server_key_version: "encryptionkey@0"
I20260809 03:37:49.009116 23346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "5ae42346c98041c6a0ed665b75da1847"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "9406594f6267e54448731ab092975f62"
server_key_iv: "4a7a5de7c70b8cd4c6d889c4e03140a8"
server_key_version: "encryptionkey@0"
I20260809 03:37:49.013289 23346 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:37:49.016002 23366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:49.017424 23346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:37:49.017544 23346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "5ae42346c98041c6a0ed665b75da1847"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "9406594f6267e54448731ab092975f62"
server_key_iv: "4a7a5de7c70b8cd4c6d889c4e03140a8"
server_key_version: "encryptionkey@0"
I20260809 03:37:49.017659 23346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:49.055734 23346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:49.059053 23346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:49.059288 23346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:49.066620 23346 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:37957
I20260809 03:37:49.066926 23438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:37957 every 8 connection(s)
I20260809 03:37:49.067687 23346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:37:49.070719 23440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:49.075304 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 23346
I20260809 03:37:49.075446 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:37:49.075740 1921 external_mini_cluster.cc:1428] Setting key be2c7365484dcf6e6259309ab8bd7548
I20260809 03:37:49.077057 23440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Bootstrap starting.
I20260809 03:37:49.079773 23440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:49.080770 23440 log.cc:824] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:49.083287 23440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: No bootstrap required, opened a new log
Aug 09 03:37:49 dist-test-slave-jq4t krb5kdc[23327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:49.088800 23440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } }
I20260809 03:37:49.089147 23440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:49.089215 23440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae42346c98041c6a0ed665b75da1847, State: Initialized, Role: FOLLOWER
I20260809 03:37:49.089803 23440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } }
I20260809 03:37:49.089984 23440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:37:49.090081 23440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:37:49.090196 23440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:49.091506 23440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } }
I20260809 03:37:49.091881 23440 leader_election.cc:302] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ae42346c98041c6a0ed665b75da1847; no voters:
I20260809 03:37:49.092272 23440 leader_election.cc:288] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:37:49.092352 23445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:37:49.092614 23445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 1 LEADER]: Becoming Leader. State: Replica: 5ae42346c98041c6a0ed665b75da1847, State: Running, Role: LEADER
I20260809 03:37:49.092661 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.077401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57814 (local address 127.1.224.126:37957)
0809 03:37:49.077942 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:49.077960 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:49.078000 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:37:49.078692 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:49.078710 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:49.079055 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:49.079349 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:49.080377 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.081747 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.082960 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.083516 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.083662 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.086645 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:49.086654 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:49.086684 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:49.086732 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:49.088932 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:49.089509 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:49.089515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:49.089520 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:49.089725 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:49.090063 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:49.090069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:49.090071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:49.090609 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:49.090771 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:49.091174 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 03:37:49.091559 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":169,"threads_started":1}
I20260809 03:37:49.092941 23445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } }
I20260809 03:37:49.093767 23440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:37:49.094220 23445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ae42346c98041c6a0ed665b75da1847. Latest consensus state: current_term: 1 leader_uuid: "5ae42346c98041c6a0ed665b75da1847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } } }
I20260809 03:37:49.094383 23445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:49.094216 23446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ae42346c98041c6a0ed665b75da1847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } } }
I20260809 03:37:49.094664 23446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:49.095024 23449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:37:49.098178 23449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:37:49.104064 23449 catalog_manager.cc:1347] Generated new cluster ID: 479838f86abf4920a3963a37dad7bb18
I20260809 03:37:49.104161 23449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:37:49.116731 23449 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:37:49.118160 23449 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:37:49.124807 23449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Generated new TSK 0
I20260809 03:37:49.125773 23449 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:37:49.159770 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:37957
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:37:49.285542 23472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:49.285841 23472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:49.285888 23472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:49.290165 23472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:49.290239 23472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:49.290302 23472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:37:49.296128 23472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:37957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:49.297420 23472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:49.298480 23472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:49.308815 23479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:49.308852 23483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:49.308984 23472 server_base.cc:1034] running on GCE node
W20260809 03:37:49.309192 23480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:49.309674 23472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:49.310408 23472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:49.311672 23472 hybrid_clock.cc:648] HybridClock initialized: now 1786246669311636 us; error 51 us; skew 500 ppm
Aug 09 03:37:49 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246669, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:49.315279 23472 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:37:49.316367 23472 webserver.cc:460] Webserver started at http://127.1.224.65:33811/ using document root <none> and password file <none>
I20260809 03:37:49.317059 23472 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:49.317179 23472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:49.317452 23472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:37:49.319818 23472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "d331db6119b043ee965a0e3e8ece4e8e"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "00bce3f9e39861b9817b9dd2b06a90b5"
server_key_iv: "665f3a907942df32e23d7048aa6d85ed"
server_key_version: "encryptionkey@0"
I20260809 03:37:49.320519 23472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "d331db6119b043ee965a0e3e8ece4e8e"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "00bce3f9e39861b9817b9dd2b06a90b5"
server_key_iv: "665f3a907942df32e23d7048aa6d85ed"
server_key_version: "encryptionkey@0"
I20260809 03:37:49.324956 23472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:37:49.327800 23496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:49.329238 23472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:37:49.329392 23472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "d331db6119b043ee965a0e3e8ece4e8e"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "00bce3f9e39861b9817b9dd2b06a90b5"
server_key_iv: "665f3a907942df32e23d7048aa6d85ed"
server_key_version: "encryptionkey@0"
I20260809 03:37:49.329525 23472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:49.344954 23472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:49.348304 23472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:49.348619 23472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:49.349314 23472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:49.350247 23472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:37:49.350337 23472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:49.350437 23472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:49.350490 23472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:49.359906 23472 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:37581
I20260809 03:37:49.359943 23643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:37581 every 8 connection(s)
I20260809 03:37:49.361369 23472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:37:49.368665 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 23472
I20260809 03:37:49.368856 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:37:49.369181 1921 external_mini_cluster.cc:1428] Setting key 2a96c9d3c9b24b93ab51b7f89a40ba9f
Aug 09 03:37:49 dist-test-slave-jq4t krb5kdc[23327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246669, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:49.378000 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.363854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:34633 (local address 127.1.224.126:37957)
0809 03:37:49.364022 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:49.364026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:49.365275 (+ 1249us) server_negotiation.cc:407] Connection header received
0809 03:37:49.366211 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:49.366214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:49.366269 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:49.366369 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:49.367704 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.368290 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.370250 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.370581 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.370736 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.374042 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:49.374046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:49.374048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:49.374075 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:49.375488 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:49.376158 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:49.376160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:49.376162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:49.376241 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:49.376714 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:49.376717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:49.376718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:49.376916 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:49.377100 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:49.377671 (+ 571us) server_negotiation.cc:299] Negotiation successful
0809 03:37:49.377818 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 03:37:49.378913 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.364278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:34633)
0809 03:37:49.365087 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:49.365135 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:49.365987 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:49.366515 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:49.366522 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:49.366946 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:49.367487 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.367502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.368467 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.368473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.370107 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.370127 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.370783 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.370789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.371033 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.371723 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:49.371749 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:49.373816 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:49.375614 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:49.375624 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:49.375645 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:49.376045 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:49.376334 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:49.376340 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:49.376345 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:49.376613 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:49.377103 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:49.377112 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:49.377402 (+ 290us) client_negotiation.cc:769] Sending connection context
0809 03:37:49.377609 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 03:37:49.377986 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":170,"threads_started":1}
I20260809 03:37:49.380520 23644 heartbeater.cc:347] Connected to a master server at 127.1.224.126:37957
I20260809 03:37:49.380823 23644 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:49.381656 23644 heartbeater.cc:511] Master 127.1.224.126:37957 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:37:49.383917 23391 ts_manager.cc:194] Registered new tserver with Master: d331db6119b043ee965a0e3e8ece4e8e (127.1.224.65:37581)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:37:49.385757 23391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:34633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:37:49.403901 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:37957
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:37:49.526410 23652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:49.526746 23652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:49.526794 23652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:49.531301 23652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:49.531383 23652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:49.531446 23652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:37:49.537446 23652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:37957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:49.538700 23652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:49.539808 23652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:49.551214 23657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:49.551303 23652 server_base.cc:1034] running on GCE node
W20260809 03:37:49.551230 23658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:49.551244 23661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:49.552170 23652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:49.552976 23652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:49.554191 23652 hybrid_clock.cc:648] HybridClock initialized: now 1786246669554166 us; error 35 us; skew 500 ppm
Aug 09 03:37:49 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246669, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:49.557646 23652 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:37:49.558754 23652 webserver.cc:460] Webserver started at http://127.1.224.66:43699/ using document root <none> and password file <none>
I20260809 03:37:49.559414 23652 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:49.559486 23652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:49.559723 23652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:37:49.562150 23652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "f06660db80de49d5b6b1ea3c9027d222"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "6c051a67f43edd4d007318acc4eb13fa"
server_key_iv: "1e3059737f0a13c42f8b7973b36810f3"
server_key_version: "encryptionkey@0"
I20260809 03:37:49.562789 23652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "f06660db80de49d5b6b1ea3c9027d222"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "6c051a67f43edd4d007318acc4eb13fa"
server_key_iv: "1e3059737f0a13c42f8b7973b36810f3"
server_key_version: "encryptionkey@0"
I20260809 03:37:49.567148 23652 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:37:49.570092 23671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:49.571552 23652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 03:37:49.571815 23652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "f06660db80de49d5b6b1ea3c9027d222"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "6c051a67f43edd4d007318acc4eb13fa"
server_key_iv: "1e3059737f0a13c42f8b7973b36810f3"
server_key_version: "encryptionkey@0"
I20260809 03:37:49.571955 23652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:49.597792 23652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:49.601102 23652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:49.601328 23652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:49.602005 23652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:49.602937 23652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:37:49.602988 23652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:49.603061 23652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:49.603101 23652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:49.613301 23652 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:42117
I20260809 03:37:49.613325 23829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:42117 every 8 connection(s)
I20260809 03:37:49.614383 23652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:37:49.622830 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 23652
I20260809 03:37:49.622943 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:37:49.623239 1921 external_mini_cluster.cc:1428] Setting key 462f304dde14f7672a593286eec139d0
Aug 09 03:37:49 dist-test-slave-jq4t krb5kdc[23327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246669, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:49.629422 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.616498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:51687 (local address 127.1.224.126:37957)
0809 03:37:49.616671 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:49.616675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:49.617806 (+ 1131us) server_negotiation.cc:407] Connection header received
0809 03:37:49.618769 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:49.618773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:49.618837 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:49.618921 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:49.620202 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.620855 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.622078 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.622323 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.622447 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.625417 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:49.625423 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:49.625430 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:49.625456 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:49.627121 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:49.627688 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:49.627697 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:49.627698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:49.627777 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:49.628154 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:49.628158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:49.628160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:49.628375 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:49.628486 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:49.629099 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 03:37:49.629240 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:37:49.630419 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.616921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:51687)
0809 03:37:49.617634 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:49.617680 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:49.618554 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:49.619107 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:49.619114 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:49.619504 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:49.620033 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.620048 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.620997 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.621006 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.621932 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.621942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.622537 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.622543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.622724 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.623434 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:49.623453 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:49.625239 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:49.627256 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:49.627263 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:49.627283 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:49.627569 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:49.627882 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:49.627886 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:49.627888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:49.628046 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:49.628533 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:49.628542 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:49.628828 (+ 286us) client_negotiation.cc:769] Sending connection context
0809 03:37:49.629042 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 03:37:49.629417 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":131,"threads_started":1}
I20260809 03:37:49.631845 23830 heartbeater.cc:347] Connected to a master server at 127.1.224.126:37957
I20260809 03:37:49.632153 23830 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:49.632755 23830 heartbeater.cc:511] Master 127.1.224.126:37957 requested a full tablet report, sending...
I20260809 03:37:49.634019 23391 ts_manager.cc:194] Registered new tserver with Master: f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117)
I20260809 03:37:49.634959 23391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:51687
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:37:49.655747 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:37957
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:37:49.779174 23838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:49.779567 23838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:49.779620 23838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:49.784042 23838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:49.784118 23838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:49.784183 23838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:37:49.790009 23838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:37957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:49.791342 23838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:49.792439 23838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:49.803550 23846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:49.803565 23845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:49.803567 23848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:49.803929 23838 server_base.cc:1034] running on GCE node
I20260809 03:37:49.804402 23838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:49.805141 23838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:49.806319 23838 hybrid_clock.cc:648] HybridClock initialized: now 1786246669806296 us; error 35 us; skew 500 ppm
Aug 09 03:37:49 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246669, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:49.809527 23838 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:37:49.810557 23838 webserver.cc:460] Webserver started at http://127.1.224.67:36597/ using document root <none> and password file <none>
I20260809 03:37:49.811211 23838 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:49.811280 23838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:49.811535 23838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:37:49.813978 23838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "351801e790714f879833f6f5f719b800"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "7cc5aab7f18d16c0577dd51772c60e47"
server_key_iv: "df23e315ca7a151dcbe1c80c1b97f7a4"
server_key_version: "encryptionkey@0"
I20260809 03:37:49.814607 23838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "351801e790714f879833f6f5f719b800"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "7cc5aab7f18d16c0577dd51772c60e47"
server_key_iv: "df23e315ca7a151dcbe1c80c1b97f7a4"
server_key_version: "encryptionkey@0"
I20260809 03:37:49.818636 23838 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:37:49.821372 23856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:49.822824 23838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:37:49.822968 23838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "351801e790714f879833f6f5f719b800"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "7cc5aab7f18d16c0577dd51772c60e47"
server_key_iv: "df23e315ca7a151dcbe1c80c1b97f7a4"
server_key_version: "encryptionkey@0"
I20260809 03:37:49.823102 23838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:49.861744 23838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:49.865669 23838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:49.865903 23838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:49.866597 23838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:49.867568 23838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:37:49.867620 23838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:49.867666 23838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:49.867720 23838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:49.877784 23838 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:44013
I20260809 03:37:49.877808 24014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:44013 every 8 connection(s)
I20260809 03:37:49.878861 23838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:37:49.885386 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 23838
I20260809 03:37:49.885521 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:37:49.885857 1921 external_mini_cluster.cc:1428] Setting key 56ef809ddba73cea7d57ff3d58ec246d
Aug 09 03:37:49 dist-test-slave-jq4t krb5kdc[23327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246669, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:49.893663 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.880996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:34695 (local address 127.1.224.126:37957)
0809 03:37:49.881193 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:49.881198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:49.882281 (+ 1083us) server_negotiation.cc:407] Connection header received
0809 03:37:49.883125 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:49.883128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:49.883178 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:49.883255 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:49.884533 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.885111 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.886806 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.887129 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.887246 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.890179 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:49.890182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:49.890185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:49.890212 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:49.891611 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:49.892136 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:49.892139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:49.892140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:49.892220 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:49.892593 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:49.892596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:49.892597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:49.892797 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:49.892878 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:49.893411 (+ 533us) server_negotiation.cc:299] Negotiation successful
0809 03:37:49.893531 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 03:37:49.894588 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.881421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:34695)
0809 03:37:49.882118 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:49.882163 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:49.882930 (+ 767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:49.883430 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:49.883437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:49.883812 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:49.884331 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.884353 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.885272 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.885276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.886672 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.886687 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.887340 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.887346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.887566 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.888296 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:49.888332 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:49.890002 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:49.891730 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:49.891735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:49.891750 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:49.892030 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:49.892305 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:49.892308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:49.892309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:49.892455 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:49.892893 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:49.892899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:49.893185 (+ 286us) client_negotiation.cc:769] Sending connection context
0809 03:37:49.893359 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 03:37:49.893652 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":114,"threads_started":1}
I20260809 03:37:49.896190 24015 heartbeater.cc:347] Connected to a master server at 127.1.224.126:37957
I20260809 03:37:49.896534 24015 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:49.897105 24015 heartbeater.cc:511] Master 127.1.224.126:37957 requested a full tablet report, sending...
I20260809 03:37:49.898300 23391 ts_manager.cc:194] Registered new tserver with Master: 351801e790714f879833f6f5f719b800 (127.1.224.67:44013)
I20260809 03:37:49.898967 23391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:34695
I20260809 03:37:49.900470 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:37:49.910698 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.902738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57822 (local address 127.1.224.126:37957)
0809 03:37:49.902920 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:49.902924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:49.903002 (+ 78us) server_negotiation.cc:407] Connection header received
0809 03:37:49.903115 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:49.903117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:49.903158 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:49.903228 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:49.903966 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.904564 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.905608 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.905925 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.906013 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.907466 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:49.907470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:49.907472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:49.907499 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:49.909028 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:49.909447 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:49.909450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:49.909451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:49.909533 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:49.909784 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:49.909786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:49.909788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:49.910033 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:49.910144 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:49.910364 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 03:37:49.910473 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 03:37:49.914179 23391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:37:49.917531 23391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:37:49.933631 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.927190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58740 (local address 127.1.224.65:37581)
0809 03:37:49.927994 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:49.928002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:49.928030 (+ 28us) server_negotiation.cc:407] Connection header received
0809 03:37:49.928119 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:49.928124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:49.928695 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:49.929239 (+ 544us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:49.929375 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.930642 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.932190 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.932863 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.932995 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.933178 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:49.933343 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 03:37:49.933453 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":92,"threads_started":1}
I20260809 03:37:49.933653 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.926995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:37581 (local address 127.0.0.1:58740)
0809 03:37:49.927521 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:49.927573 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:49.927717 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:49.928866 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:49.928872 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:49.928898 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:49.929084 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.929092 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.930775 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.930778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.932063 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.932071 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.932994 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.932997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.933113 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.933155 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 03:37:49.933305 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 03:37:49.933461 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":136,"thread_start_us":107,"threads_started":1}
I20260809 03:37:49.933766 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.927809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:42117 (local address 127.0.0.1:50982)
0809 03:37:49.928426 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:49.928441 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:49.928553 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:49.929277 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:49.929280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:49.929297 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:49.929379 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.929384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.930803 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.930805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.932057 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.932069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.933387 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.933391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.933472 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.933492 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 03:37:49.933564 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 03:37:49.933627 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":68,"thread_start_us":97,"threads_started":1}
I20260809 03:37:49.934365 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.927996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50982 (local address 127.1.224.66:42117)
0809 03:37:49.928459 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:49.928498 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:49.928551 (+ 53us) server_negotiation.cc:407] Connection header received
0809 03:37:49.928719 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:49.928724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:49.928916 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:49.929087 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:49.929517 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.930642 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.932202 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.933022 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.933172 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.933328 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:49.934096 (+ 768us) server_negotiation.cc:299] Negotiation successful
0809 03:37:49.934222 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":137,"threads_started":1}
I20260809 03:37:49.934542 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.928241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.1.224.67:44013)
0809 03:37:49.928692 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:49.928703 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:49.928724 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:37:49.928801 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:49.928807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:49.929009 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:49.929199 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:49.929443 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.930246 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.931950 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.932911 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.933960 (+ 1049us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.934135 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:49.934182 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 03:37:49.934266 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":99,"threads_started":1}
I20260809 03:37:49.933652 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.927205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:44013 (local address 127.0.0.1:34626)
0809 03:37:49.927802 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:49.927818 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:49.927915 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:49.929174 (+ 1259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:49.929178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:49.929197 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:49.929313 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.929322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.930383 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.930387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.931818 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.931830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.933110 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.933117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.933211 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:49.933234 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:37:49.933288 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 03:37:49.933455 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":3,"thread_start_us":120,"threads_started":1}
I20260809 03:37:49.937263 23560 tablet_service.cc:1467] Processing CreateTablet for tablet f733d2df6fd04a12b4634cd030c27caf (DEFAULT_TABLE table=test-table [id=7353c205726e40debe4fc380b7c69e02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:37:49.937222 23734 tablet_service.cc:1467] Processing CreateTablet for tablet f733d2df6fd04a12b4634cd030c27caf (DEFAULT_TABLE table=test-table [id=7353c205726e40debe4fc380b7c69e02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:37:49.938346 23560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f733d2df6fd04a12b4634cd030c27caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:49.938364 23734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f733d2df6fd04a12b4634cd030c27caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:49.939347 23927 tablet_service.cc:1467] Processing CreateTablet for tablet f733d2df6fd04a12b4634cd030c27caf (DEFAULT_TABLE table=test-table [id=7353c205726e40debe4fc380b7c69e02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:37:49.940508 23927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f733d2df6fd04a12b4634cd030c27caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:49.945277 24036 tablet_bootstrap.cc:492] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: Bootstrap starting.
I20260809 03:37:49.946363 24037 tablet_bootstrap.cc:492] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222: Bootstrap starting.
I20260809 03:37:49.947659 24038 tablet_bootstrap.cc:492] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800: Bootstrap starting.
I20260809 03:37:49.948277 24036 tablet_bootstrap.cc:654] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:49.949047 24037 tablet_bootstrap.cc:654] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:49.949337 24036 log.cc:824] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:49.949623 24038 tablet_bootstrap.cc:654] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:49.949900 24037 log.cc:824] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:49.950904 24038 log.cc:824] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:49.951975 24036 tablet_bootstrap.cc:492] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: No bootstrap required, opened a new log
I20260809 03:37:49.952135 24037 tablet_bootstrap.cc:492] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222: No bootstrap required, opened a new log
I20260809 03:37:49.952195 24036 ts_tablet_manager.cc:1397] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 03:37:49.952346 24037 ts_tablet_manager.cc:1397] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 03:37:49.952896 24038 tablet_bootstrap.cc:492] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800: No bootstrap required, opened a new log
I20260809 03:37:49.953076 24038 ts_tablet_manager.cc:1397] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260809 03:37:49.955267 24037 raft_consensus.cc:348] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } }
I20260809 03:37:49.955518 24037 raft_consensus.cc:374] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:49.955574 24037 raft_consensus.cc:729] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06660db80de49d5b6b1ea3c9027d222, State: Initialized, Role: FOLLOWER
I20260809 03:37:49.956050 24037 consensus_queue.cc:260] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } }
I20260809 03:37:49.956786 23830 heartbeater.cc:503] Master 127.1.224.126:37957 was elected leader, sending a full tablet report...
I20260809 03:37:49.956941 24036 raft_consensus.cc:348] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } }
I20260809 03:37:49.956941 24038 raft_consensus.cc:348] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } }
I20260809 03:37:49.957185 24038 raft_consensus.cc:374] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:49.957185 24036 raft_consensus.cc:374] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:49.957257 24036 raft_consensus.cc:729] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d331db6119b043ee965a0e3e8ece4e8e, State: Initialized, Role: FOLLOWER
I20260809 03:37:49.957257 24038 raft_consensus.cc:729] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351801e790714f879833f6f5f719b800, State: Initialized, Role: FOLLOWER
I20260809 03:37:49.957697 24037 ts_tablet_manager.cc:1428] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 03:37:49.957679 24036 consensus_queue.cc:260] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } }
I20260809 03:37:49.957716 24038 consensus_queue.cc:260] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } }
I20260809 03:37:49.958364 23644 heartbeater.cc:503] Master 127.1.224.126:37957 was elected leader, sending a full tablet report...
I20260809 03:37:49.958669 24038 ts_tablet_manager.cc:1428] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:37:49.959311 24015 heartbeater.cc:503] Master 127.1.224.126:37957 was elected leader, sending a full tablet report...
I20260809 03:37:49.960645 24036 ts_tablet_manager.cc:1428] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 03:37:49.994992 24046 raft_consensus.cc:482] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:37:49.995317 24046 raft_consensus.cc:504] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } }
I20260809 03:37:49.996601 24046 leader_election.cc:288] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 351801e790714f879833f6f5f719b800 (127.1.224.67:44013), f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117)
I20260809 03:37:50.001210 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.997507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:36639 (local address 127.1.224.66:42117)
0809 03:37:49.997660 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:49.997665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:49.997710 (+ 45us) server_negotiation.cc:407] Connection header received
0809 03:37:49.997865 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:49.997868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:49.997923 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:49.998002 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:49.998290 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.998777 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.000001 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.000541 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.000645 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:50.000693 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:50.000929 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 03:37:50.001033 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:37:50.001255 24048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.997340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:42117 (local address 127.1.224.65:36639)
0809 03:37:49.997627 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:49.997648 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:49.997751 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:49.998017 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:49.998019 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:49.998033 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:49.998168 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.998175 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.998923 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.998925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:49.999864 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.999874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.000663 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.000667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:50.000779 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:50.000820 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 03:37:50.000895 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 03:37:50.001032 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":75,"thread_start_us":102,"threads_started":1}
I20260809 03:37:50.001420 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.997014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:58815 (local address 127.1.224.67:44013)
0809 03:37:49.997174 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:49.997178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:49.997331 (+ 153us) server_negotiation.cc:407] Connection header received
0809 03:37:49.997549 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:49.997556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:49.997616 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:49.997672 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:49.998201 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.998762 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.000203 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.000725 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.000851 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:50.000914 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:50.001207 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 03:37:50.001302 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 03:37:50.001623 24047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:49.996881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:44013 (local address 127.1.224.65:58815)
0809 03:37:49.997216 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:49.997234 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:49.997366 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:49.997874 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:49.997878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:49.997896 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:49.998060 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:49.998068 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:49.998887 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:49.998893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:50.000061 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:50.000072 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.000850 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.000853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:50.000979 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:50.001053 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 03:37:50.001165 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 03:37:50.001246 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":2,"thread_start_us":146,"threads_started":1}
I20260809 03:37:50.002064 23762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f733d2df6fd04a12b4634cd030c27caf" candidate_uuid: "d331db6119b043ee965a0e3e8ece4e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f06660db80de49d5b6b1ea3c9027d222" is_pre_election: true
I20260809 03:37:50.002480 23762 raft_consensus.cc:2454] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d331db6119b043ee965a0e3e8ece4e8e in term 0.
I20260809 03:37:50.002882 23959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f733d2df6fd04a12b4634cd030c27caf" candidate_uuid: "d331db6119b043ee965a0e3e8ece4e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351801e790714f879833f6f5f719b800" is_pre_election: true
I20260809 03:37:50.003090 23497 leader_election.cc:302] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d331db6119b043ee965a0e3e8ece4e8e, f06660db80de49d5b6b1ea3c9027d222; no voters:
I20260809 03:37:50.003259 23959 raft_consensus.cc:2454] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d331db6119b043ee965a0e3e8ece4e8e in term 0.
I20260809 03:37:50.003476 24046 raft_consensus.cc:2781] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:37:50.003598 24046 raft_consensus.cc:482] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:37:50.003690 24046 raft_consensus.cc:3037] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:50.005009 24046 raft_consensus.cc:504] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } }
I20260809 03:37:50.005424 24046 leader_election.cc:288] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 1 election: Requested vote from peers 351801e790714f879833f6f5f719b800 (127.1.224.67:44013), f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117)
I20260809 03:37:50.005900 23762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f733d2df6fd04a12b4634cd030c27caf" candidate_uuid: "d331db6119b043ee965a0e3e8ece4e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f06660db80de49d5b6b1ea3c9027d222"
I20260809 03:37:50.005923 23959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f733d2df6fd04a12b4634cd030c27caf" candidate_uuid: "d331db6119b043ee965a0e3e8ece4e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351801e790714f879833f6f5f719b800"
I20260809 03:37:50.006070 23959 raft_consensus.cc:3037] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:50.006075 23762 raft_consensus.cc:3037] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:50.007524 23959 raft_consensus.cc:2454] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d331db6119b043ee965a0e3e8ece4e8e in term 1.
I20260809 03:37:50.007534 23762 raft_consensus.cc:2454] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d331db6119b043ee965a0e3e8ece4e8e in term 1.
I20260809 03:37:50.007844 23497 leader_election.cc:302] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 351801e790714f879833f6f5f719b800, d331db6119b043ee965a0e3e8ece4e8e; no voters:
I20260809 03:37:50.008064 24046 raft_consensus.cc:2781] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:37:50.008350 24046 raft_consensus.cc:686] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 1 LEADER]: Becoming Leader. State: Replica: d331db6119b043ee965a0e3e8ece4e8e, State: Running, Role: LEADER
I20260809 03:37:50.008857 24046 consensus_queue.cc:237] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } }
I20260809 03:37:50.013003 23390 catalog_manager.cc:5495] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e reported cstate change: term changed from 0 to 1, leader changed from <none> to d331db6119b043ee965a0e3e8ece4e8e (127.1.224.65). New cstate: current_term: 1 leader_uuid: "d331db6119b043ee965a0e3e8ece4e8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } health_report { overall_health: UNKNOWN } } }
I20260809 03:37:50.035948 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:50.031871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58742 (local address 127.1.224.65:37581)
0809 03:37:50.032016 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:50.032022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:50.032036 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:37:50.032157 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:50.032160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:50.032220 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:50.032348 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:50.032707 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.033385 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.034372 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.034611 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.034705 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:50.035092 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:50.035587 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:50.035694 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:50.035755 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 03:37:50.035819 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:37:50.043062 23959 raft_consensus.cc:1261] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800 [term 1 FOLLOWER]: Refusing update from remote peer d331db6119b043ee965a0e3e8ece4e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:37:50.043054 23762 raft_consensus.cc:1261] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222 [term 1 FOLLOWER]: Refusing update from remote peer d331db6119b043ee965a0e3e8ece4e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:37:50.043982 24049 consensus_queue.cc:1035] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:37:50.044306 24046 consensus_queue.cc:1035] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:37:50.053721 24056 mvcc.cc:205] Tried to move back new op lower bound from 7316466360486461440 to 7316466360360058880. Current Snapshot: MvccSnapshot[applied={T|T < 7316466360486461440}]
I20260809 03:37:50.063176 23388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:37:50.063390 23388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:37:50.066056 23388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Sending DeleteTablet for 3 replicas of tablet f733d2df6fd04a12b4634cd030c27caf
I20260809 03:37:50.067028 23560 tablet_service.cc:1514] Processing DeleteTablet for tablet f733d2df6fd04a12b4634cd030c27caf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:37:50 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:58740
I20260809 03:37:50.067606 24067 tablet_replica.cc:330] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: stopping tablet replica
I20260809 03:37:50.067956 24067 raft_consensus.cc:2229] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:37:50.068404 24067 raft_consensus.cc:2258] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:37:50.069913 24067 ts_tablet_manager.cc:1905] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:37:50.072979 24067 ts_tablet_manager.cc:1918] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:37:50.073067 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 23472
I20260809 03:37:50.073092 24067 log.cc:1196] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal/wals/f733d2df6fd04a12b4634cd030c27caf
I20260809 03:37:50.073133 23927 tablet_service.cc:1514] Processing DeleteTablet for tablet f733d2df6fd04a12b4634cd030c27caf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:37:50 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:34626
I20260809 03:37:50.087841 23734 tablet_service.cc:1514] Processing DeleteTablet for tablet f733d2df6fd04a12b4634cd030c27caf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:37:50 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:50982
I20260809 03:37:50.088536 24071 tablet_replica.cc:330] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222: stopping tablet replica
I20260809 03:37:50.088567 24070 tablet_replica.cc:330] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800: stopping tablet replica
I20260809 03:37:50.089001 24070 raft_consensus.cc:2229] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:37:50.089015 24071 raft_consensus.cc:2229] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:37:50.089416 24070 raft_consensus.cc:2258] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:37:50.089525 24071 raft_consensus.cc:2258] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:37:50.091223 24070 ts_tablet_manager.cc:1905] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:37:50.091223 24071 ts_tablet_manager.cc:1905] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:37:50.094116 24070 ts_tablet_manager.cc:1918] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:37:50.094223 24070 log.cc:1196] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/f733d2df6fd04a12b4634cd030c27caf
I20260809 03:37:50.094632 24070 ts_tablet_manager.cc:1939] T f733d2df6fd04a12b4634cd030c27caf P 351801e790714f879833f6f5f719b800: Deleting consensus metadata
I20260809 03:37:50.095515 24071 ts_tablet_manager.cc:1918] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:37:50.095676 24071 log.cc:1196] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/f733d2df6fd04a12b4634cd030c27caf
I20260809 03:37:50.096149 24071 ts_tablet_manager.cc:1939] T f733d2df6fd04a12b4634cd030c27caf P f06660db80de49d5b6b1ea3c9027d222: Deleting consensus metadata
I20260809 03:37:50.096210 23367 catalog_manager.cc:4841] TS 351801e790714f879833f6f5f719b800 (127.1.224.67:44013): tablet f733d2df6fd04a12b4634cd030c27caf (table test-table [id=7353c205726e40debe4fc380b7c69e02]) successfully deleted
W20260809 03:37:50.096763 23373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:37581 (error 108)
I20260809 03:37:50.097070 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 23652
I20260809 03:37:50.103052 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:50.102764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:37581 (local address 127.0.0.1:58750)
0809 03:37:50.102887 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:50.102966 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:37581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 03:37:50.103309 23373 catalog_manager.cc:4584] TS d331db6119b043ee965a0e3e8ece4e8e (127.1.224.65:37581): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f733d2df6fd04a12b4634cd030c27caf: Network error: Client connection negotiation failed: client connection to 127.1.224.65:37581: connect: Connection refused (error 111)
I20260809 03:37:50.109050 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 23838
I20260809 03:37:50.115453 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:50.109264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:42117 (local address 127.0.0.1:50988)
0809 03:37:50.115178 (+ 5914us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:50.115300 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.66:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5757}
I20260809 03:37:50.127897 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 23346
I20260809 03:37:50.147006 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37957
--webserver_interface=127.1.224.126
--webserver_port=39549
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:37957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:37:50.269667 24073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:50.270056 24073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:50.270179 24073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:50.274466 24073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:37:50.274581 24073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:50.274621 24073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:50.274665 24073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:37:50.274703 24073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:37:50.280442 24073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:37957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=39549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:50.281772 24073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:50.282898 24073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:50.298331 24082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:50.298331 24084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:50.298494 24073 server_base.cc:1034] running on GCE node
W20260809 03:37:50.298362 24086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:50.299499 24073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:50.300825 24073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:50.302011 24073 hybrid_clock.cc:648] HybridClock initialized: now 1786246670301964 us; error 59 us; skew 500 ppm
Aug 09 03:37:50 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:50.305150 24073 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:37:50.306173 24073 webserver.cc:460] Webserver started at http://127.1.224.126:39549/ using document root <none> and password file <none>
I20260809 03:37:50.306845 24073 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:50.306972 24073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:50.311385 24073 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:37:50.314010 24097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:50.315446 24073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 03:37:50.315634 24073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "5ae42346c98041c6a0ed665b75da1847"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "9406594f6267e54448731ab092975f62"
server_key_iv: "4a7a5de7c70b8cd4c6d889c4e03140a8"
server_key_version: "encryptionkey@0"
I20260809 03:37:50.316356 24073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:50.335582 24073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:50.338953 24073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:50.339205 24073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:50.347414 24073 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:37957
I20260809 03:37:50.347735 24165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:37957 every 8 connection(s)
I20260809 03:37:50.348757 24073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:37:50.353690 24166 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:37:50.355453 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 24073
I20260809 03:37:50.355525 24166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Bootstrap starting.
I20260809 03:37:50.355952 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:37581
--local_ip_for_outbound_sockets=127.1.224.65
--tserver_master_addrs=127.1.224.126:37957
--webserver_port=33811
--webserver_interface=127.1.224.65
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:37:50.381781 24166 log.cc:824] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:50.395380 24166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:37:50.395949 24166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Bootstrap complete.
I20260809 03:37:50.402184 24166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } }
I20260809 03:37:50.402783 24166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae42346c98041c6a0ed665b75da1847, State: Initialized, Role: FOLLOWER
I20260809 03:37:50.403407 24166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } }
I20260809 03:37:50.403578 24166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:37:50.403664 24166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:37:50.403770 24166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:37:50.406852 24166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } }
I20260809 03:37:50.407294 24166 leader_election.cc:302] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ae42346c98041c6a0ed665b75da1847; no voters:
I20260809 03:37:50.407701 24166 leader_election.cc:288] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 03:37:50.407871 24171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:37:50.408175 24171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 2 LEADER]: Becoming Leader. State: Replica: 5ae42346c98041c6a0ed665b75da1847, State: Running, Role: LEADER
I20260809 03:37:50.408571 24171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } }
I20260809 03:37:50.409231 24166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:37:50.410243 24173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ae42346c98041c6a0ed665b75da1847. Latest consensus state: current_term: 2 leader_uuid: "5ae42346c98041c6a0ed665b75da1847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } } }
I20260809 03:37:50.410403 24173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:50.410717 24172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5ae42346c98041c6a0ed665b75da1847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } } }
I20260809 03:37:50.410816 24172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:50.410945 24181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:37:50.417627 24181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:37:50.418465 24181 catalog_manager.cc:1259] Loaded cluster ID: 479838f86abf4920a3963a37dad7bb18
I20260809 03:37:50.418542 24181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:37:50.420064 24181 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 03:37:50.425092 24181 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 03:37:50.431643 24181 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:37:50.432685 24181 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Loaded TSK: 0
I20260809 03:37:50.434031 24181 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:37:50.503821 24168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:50.504169 24168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:50.504256 24168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:50.509407 24168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:50.509529 24168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:50.509627 24168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:37:50.516765 24168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:37581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=33811
--tserver_master_addrs=127.1.224.126:37957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:50.518075 24168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:50.519272 24168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:50.531102 24199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:50.531102 24198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:50.531109 24203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:50.531597 24168 server_base.cc:1034] running on GCE node
I20260809 03:37:50.532152 24168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:50.532944 24168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:50.534145 24168 hybrid_clock.cc:648] HybridClock initialized: now 1786246670534116 us; error 50 us; skew 500 ppm
Aug 09 03:37:50 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:50.537612 24168 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:37:50.538650 24168 webserver.cc:460] Webserver started at http://127.1.224.65:33811/ using document root <none> and password file <none>
I20260809 03:37:50.539294 24168 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:50.539423 24168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:50.543975 24168 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:37:50.546674 24210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:50.548128 24168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:37:50.548292 24168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "d331db6119b043ee965a0e3e8ece4e8e"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "00bce3f9e39861b9817b9dd2b06a90b5"
server_key_iv: "665f3a907942df32e23d7048aa6d85ed"
server_key_version: "encryptionkey@0"
I20260809 03:37:50.548982 24168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:50.569487 24168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:50.572853 24168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:50.573087 24168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:50.573767 24168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:50.574957 24220 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260809 03:37:50.579267 24220 ts_tablet_manager.cc:1852] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260809 03:37:50.579682 24220 ts_tablet_manager.cc:1905] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:37:50.583870 24220 ts_tablet_manager.cc:1918] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:37:50.583984 24220 log.cc:1196] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal/wals/f733d2df6fd04a12b4634cd030c27caf
I20260809 03:37:50.584384 24220 ts_tablet_manager.cc:1939] T f733d2df6fd04a12b4634cd030c27caf P d331db6119b043ee965a0e3e8ece4e8e: Deleting consensus metadata
I20260809 03:37:50.584771 24168 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260809 03:37:50.584856 24168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.001s sys 0.000s
I20260809 03:37:50.584972 24168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:50.585016 24168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:50.594846 24168 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:37581
I20260809 03:37:50.594887 24373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:37581 every 8 connection(s)
I20260809 03:37:50.595765 24168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:37:50.596186 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 24168
I20260809 03:37:50.596728 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:42117
--local_ip_for_outbound_sockets=127.1.224.66
--tserver_master_addrs=127.1.224.126:37957
--webserver_port=43699
--webserver_interface=127.1.224.66
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:37:50 dist-test-slave-jq4t krb5kdc[23327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:50.616305 24379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:50.598322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:60883)
0809 03:37:50.598931 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:50.598996 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:50.600249 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:50.601692 (+ 1443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:50.601699 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:50.602137 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:50.602742 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:50.602757 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.604973 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.604976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:50.605879 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:50.605890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.607109 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.607112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:50.607262 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:50.607997 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:50.608015 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:50.610043 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:50.612810 (+ 2767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:50.612816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:50.612830 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:50.613148 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:50.613581 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:50.613584 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:50.613585 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:50.613741 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:50.614589 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:50.614597 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:50.614952 (+ 355us) client_negotiation.cc:769] Sending connection context
0809 03:37:50.615136 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 03:37:50.615447 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":148,"threads_started":1}
I20260809 03:37:50.616530 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:50.598069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:60883 (local address 127.1.224.126:37957)
0809 03:37:50.598624 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:50.598645 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:50.599163 (+ 518us) server_negotiation.cc:407] Connection header received
0809 03:37:50.600755 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:50.600773 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:50.601255 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:50.601632 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:50.602951 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.604815 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.606033 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.606898 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.607144 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:50.610312 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:50.610322 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:50.610346 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:50.610390 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:50.612659 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:50.613278 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:50.613285 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:50.613292 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:50.613460 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:50.613843 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:50.613846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:50.613848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:50.614384 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:50.614544 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:50.615296 (+ 752us) server_negotiation.cc:299] Negotiation successful
0809 03:37:50.615633 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":133,"threads_started":1}
I20260809 03:37:50.618845 24374 heartbeater.cc:347] Connected to a master server at 127.1.224.126:37957
I20260809 03:37:50.619135 24374 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:50.619788 24374 heartbeater.cc:511] Master 127.1.224.126:37957 requested a full tablet report, sending...
I20260809 03:37:50.621778 24117 ts_manager.cc:194] Registered new tserver with Master: d331db6119b043ee965a0e3e8ece4e8e (127.1.224.65:37581)
I20260809 03:37:50.623095 24117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:60883
W20260809 03:37:50.724587 24376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:50.724900 24376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:50.724977 24376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:50.729228 24376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:50.729302 24376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:50.729394 24376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:37:50.734915 24376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:42117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=43699
--tserver_master_addrs=127.1.224.126:37957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:50.736083 24376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:50.737234 24376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:50.747561 24392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:50.747587 24387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:50.747568 24388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:50.747898 24376 server_base.cc:1034] running on GCE node
I20260809 03:37:50.748389 24376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:50.749131 24376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:50.750325 24376 hybrid_clock.cc:648] HybridClock initialized: now 1786246670750300 us; error 36 us; skew 500 ppm
Aug 09 03:37:50 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:50.753437 24376 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:37:50.754469 24376 webserver.cc:460] Webserver started at http://127.1.224.66:43699/ using document root <none> and password file <none>
I20260809 03:37:50.755050 24376 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:50.755131 24376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:50.759866 24376 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:37:50.762096 24402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:50.763319 24376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 03:37:50.763453 24376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "f06660db80de49d5b6b1ea3c9027d222"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "6c051a67f43edd4d007318acc4eb13fa"
server_key_iv: "1e3059737f0a13c42f8b7973b36810f3"
server_key_version: "encryptionkey@0"
I20260809 03:37:50.764035 24376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:50.777813 24376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:50.780870 24376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:50.781087 24376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:50.781724 24376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:50.782604 24376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:37:50.782653 24376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:50.782724 24376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:50.782763 24376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:50.792171 24376 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:42117
I20260809 03:37:50.792191 24562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:42117 every 8 connection(s)
I20260809 03:37:50.793426 24376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:37:50.796161 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 24376
I20260809 03:37:50.796751 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:44013
--local_ip_for_outbound_sockets=127.1.224.67
--tserver_master_addrs=127.1.224.126:37957
--webserver_port=36597
--webserver_interface=127.1.224.67
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:37:50 dist-test-slave-jq4t krb5kdc[23327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:50.810467 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:50.795342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:41837 (local address 127.1.224.126:37957)
0809 03:37:50.795494 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:50.795498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:50.796740 (+ 1242us) server_negotiation.cc:407] Connection header received
0809 03:37:50.798178 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:50.798181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:50.798236 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:50.798304 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:50.799743 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.800588 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.802508 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.802754 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.802844 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:50.806169 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:50.806173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:50.806175 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:50.806204 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:50.807716 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:50.808412 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:50.808415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:50.808416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:50.808546 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:50.809000 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:50.809003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:50.809004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:50.809244 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:50.809320 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:50.810106 (+ 786us) server_negotiation.cc:299] Negotiation successful
0809 03:37:50.810329 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:37:50.811854 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:50.795679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:41837)
0809 03:37:50.796551 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:50.796615 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:50.797909 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:50.798547 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:50.798558 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:50.799037 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:50.799559 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:50.799573 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.800734 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.800740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:50.802371 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:50.802387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.802943 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.802950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:50.803207 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:50.803910 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:50.803939 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:50.805967 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:50.807849 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:50.807856 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:50.807876 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:50.808288 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:50.808658 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:50.808661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:50.808664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:50.808887 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:50.809362 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:50.809370 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:50.809803 (+ 433us) client_negotiation.cc:769] Sending connection context
0809 03:37:50.810067 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 03:37:50.810621 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":123,"threads_started":1}
I20260809 03:37:50.813277 24563 heartbeater.cc:347] Connected to a master server at 127.1.224.126:37957
I20260809 03:37:50.813562 24563 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:50.814137 24563 heartbeater.cc:511] Master 127.1.224.126:37957 requested a full tablet report, sending...
I20260809 03:37:50.815466 24117 ts_manager.cc:194] Registered new tserver with Master: f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117)
I20260809 03:37:50.816264 24117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:41837
W20260809 03:37:50.926436 24568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:50.926761 24568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:50.926808 24568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:50.931313 24568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:50.931399 24568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:50.931468 24568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:37:50.937238 24568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:44013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=36597
--tserver_master_addrs=127.1.224.126:37957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:50.938410 24568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:50.939467 24568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:37:50.949730 24568 server_base.cc:1034] running on GCE node
W20260809 03:37:50.949694 24577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:50.949676 24576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:50.949671 24579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:50.950605 24568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:50.951246 24568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:50.952423 24568 hybrid_clock.cc:648] HybridClock initialized: now 1786246670952408 us; error 29 us; skew 500 ppm
Aug 09 03:37:50 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:50.955484 24568 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:37:50.956437 24568 webserver.cc:460] Webserver started at http://127.1.224.67:36597/ using document root <none> and password file <none>
I20260809 03:37:50.957093 24568 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:50.957176 24568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:50.961366 24568 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 03:37:50.963665 24588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:50.965199 24568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:37:50.965372 24568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "351801e790714f879833f6f5f719b800"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "7cc5aab7f18d16c0577dd51772c60e47"
server_key_iv: "df23e315ca7a151dcbe1c80c1b97f7a4"
server_key_version: "encryptionkey@0"
I20260809 03:37:50.965970 24568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:50.978952 24568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:50.982156 24568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:50.982388 24568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:50.982977 24568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:50.983801 24568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:37:50.983850 24568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:50.983893 24568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:50.983914 24568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:50.993140 24568 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:44013
I20260809 03:37:50.993162 24744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:44013 every 8 connection(s)
I20260809 03:37:50.994253 24568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:37:50.995653 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 24568
Aug 09 03:37:51 dist-test-slave-jq4t krb5kdc[23327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:37:51.005625 24748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:50.996455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57832 (local address 127.1.224.126:37957)
0809 03:37:50.996771 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:50.996776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:50.996793 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:37:50.996922 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:50.996928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:50.996995 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:50.997100 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:50.998135 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:50.998722 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:50.999921 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.000147 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.000227 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.002056 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:51.002060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:51.002062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:51.002087 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:51.003802 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:51.004266 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:51.004268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:51.004270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:51.004350 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:51.004691 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:51.004694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:51.004696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:51.004897 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:51.005137 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:51.005295 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 03:37:51.005408 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":102,"threads_started":1}
I20260809 03:37:51.009819 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:50.996166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:38367 (local address 127.1.224.126:37957)
0809 03:37:50.996316 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:50.996320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:50.997773 (+ 1453us) server_negotiation.cc:407] Connection header received
0809 03:37:50.998743 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:50.998748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:50.998805 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:50.998882 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:51.000353 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.001066 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.002640 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.002890 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.003005 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.005805 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:51.005808 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:51.005811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:51.005836 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:51.007457 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:51.007986 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:51.007989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:51.007990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:51.008068 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:51.008472 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:51.008569 (+ 97us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:51.008574 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:51.008785 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:51.008929 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:51.009571 (+ 642us) server_negotiation.cc:299] Negotiation successful
0809 03:37:51.009688 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:37:51.010797 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:50.996713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:38367)
0809 03:37:50.997590 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:50.997654 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:50.998547 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:50.999092 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:50.999102 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:50.999567 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:51.000172 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:51.000208 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.001205 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.001209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:51.002513 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:51.002524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.003052 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.003056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:51.003245 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.003787 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:51.003804 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:51.005639 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:51.007603 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:51.007608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:51.007621 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:51.007885 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:51.008182 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:51.008185 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:51.008186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:51.008350 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:51.008945 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:51.008951 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:51.009229 (+ 278us) client_negotiation.cc:769] Sending connection context
0809 03:37:51.009410 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 03:37:51.009751 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":150,"threads_started":1}
I20260809 03:37:51.012033 24745 heartbeater.cc:347] Connected to a master server at 127.1.224.126:37957
I20260809 03:37:51.012318 24745 heartbeater.cc:464] Registering TS with master...
I20260809 03:37:51.012882 24745 heartbeater.cc:511] Master 127.1.224.126:37957 requested a full tablet report, sending...
I20260809 03:37:51.014066 24117 ts_manager.cc:194] Registered new tserver with Master: 351801e790714f879833f6f5f719b800 (127.1.224.67:44013)
I20260809 03:37:51.014667 24117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:38367
I20260809 03:37:51.019616 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:37:51.023998 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.020275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57844 (local address 127.1.224.126:37957)
0809 03:37:51.020418 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:51.020421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:51.020693 (+ 272us) server_negotiation.cc:407] Connection header received
0809 03:37:51.020826 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:51.020829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:51.020937 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:51.021038 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:51.021286 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.021818 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.022673 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.022909 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.022999 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.023268 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:51.023679 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:51.023770 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:51.023828 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 03:37:51.023877 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:37:51.024791 24117 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:37:51.028203 24117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:37:51.044083 24757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.037382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:42117 (local address 127.0.0.1:50996)
0809 03:37:51.037896 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:51.037952 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:51.038096 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:51.039292 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:51.039299 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:51.039324 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:51.039515 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:51.039524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.040855 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.040862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:51.042233 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:51.042246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.043408 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.043412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:51.043552 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.043641 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 03:37:51.043708 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 03:37:51.043907 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":114,"thread_start_us":176,"threads_started":1}
I20260809 03:37:51.044103 24764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.038514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.1.224.67:44013)
0809 03:37:51.038842 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:51.038853 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:51.038872 (+ 19us) server_negotiation.cc:407] Connection header received
0809 03:37:51.039109 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:51.039116 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:51.039301 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:51.039447 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:51.039862 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.041146 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.042618 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.043206 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.043354 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.043536 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:51.043870 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 03:37:51.043975 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":112,"threads_started":1}
I20260809 03:37:51.044092 24758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.037598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.1.224.66:42117)
0809 03:37:51.038744 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:51.038761 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:51.038790 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:37:51.038876 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:51.038882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:51.039108 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:51.039673 (+ 565us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:51.039860 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.040716 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.042380 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.043256 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.043610 (+ 354us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.043771 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:51.043830 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 03:37:51.043927 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1021,"thread_start_us":153,"threads_started":1}
I20260809 03:37:51.044703 24760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.037735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:37581 (local address 127.0.0.1:58766)
0809 03:37:51.038352 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:51.038369 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:51.038462 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:51.039538 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:51.039544 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:51.039567 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:51.039660 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:51.039665 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.041309 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.041312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:51.042537 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:51.042547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.043586 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.043589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:51.043685 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.043707 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 03:37:51.043761 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 03:37:51.043886 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":84,"thread_start_us":107,"threads_started":1}
I20260809 03:37:51.044994 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.038230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58766 (local address 127.1.224.65:37581)
0809 03:37:51.039007 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:51.039018 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:51.039039 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:37:51.039144 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:51.039149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:51.039356 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:51.039810 (+ 454us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:51.039979 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.041146 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.042675 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.043444 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.044007 (+ 563us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.044638 (+ 631us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:51.044704 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 03:37:51.044814 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":174,"threads_started":1}
I20260809 03:37:51.046394 24279 tablet_service.cc:1467] Processing CreateTablet for tablet 5909470c8c68403aa38cb33505951124 (DEFAULT_TABLE table=test-table [id=7f8cef9aed1340779285d66a1b6f615b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:37:51.047465 24279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5909470c8c68403aa38cb33505951124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:51.047627 24470 tablet_service.cc:1467] Processing CreateTablet for tablet 5909470c8c68403aa38cb33505951124 (DEFAULT_TABLE table=test-table [id=7f8cef9aed1340779285d66a1b6f615b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:37:51.048631 24470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5909470c8c68403aa38cb33505951124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:51.052008 24220 tablet_bootstrap.cc:492] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: Bootstrap starting.
I20260809 03:37:51.052316 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.038249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:44013 (local address 127.0.0.1:34630)
0809 03:37:51.038574 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:51.038591 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:51.038694 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:51.039611 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:51.039615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:51.039635 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:51.039719 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:51.039727 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.041341 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.041344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:51.042488 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:51.042498 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.043353 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.043356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:51.043491 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.043588 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 03:37:51.043712 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 03:37:51.044979 (+ 1267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":124,"thread_start_us":98,"threads_started":1}
I20260809 03:37:51.054826 24220 tablet_bootstrap.cc:654] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:51.055766 24220 log.cc:824] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:51.056006 24649 tablet_service.cc:1467] Processing CreateTablet for tablet 5909470c8c68403aa38cb33505951124 (DEFAULT_TABLE table=test-table [id=7f8cef9aed1340779285d66a1b6f615b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:37:51.057268 24649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5909470c8c68403aa38cb33505951124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:37:51.058022 24220 tablet_bootstrap.cc:492] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: No bootstrap required, opened a new log
I20260809 03:37:51.058226 24220 ts_tablet_manager.cc:1397] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:37:51.058374 24767 tablet_bootstrap.cc:492] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: Bootstrap starting.
I20260809 03:37:51.061434 24767 tablet_bootstrap.cc:654] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:51.062237 24767 log.cc:824] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:51.062288 24220 raft_consensus.cc:348] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:51.062592 24220 raft_consensus.cc:374] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:51.062665 24220 raft_consensus.cc:729] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d331db6119b043ee965a0e3e8ece4e8e, State: Initialized, Role: FOLLOWER
I20260809 03:37:51.063194 24220 consensus_queue.cc:260] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:51.063948 24374 heartbeater.cc:503] Master 127.1.224.126:37957 was elected leader, sending a full tablet report...
I20260809 03:37:51.064381 24220 ts_tablet_manager.cc:1428] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260809 03:37:51.065907 24771 tablet_bootstrap.cc:492] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800: Bootstrap starting.
I20260809 03:37:51.068575 24771 tablet_bootstrap.cc:654] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800: Neither blocks nor log segments found. Creating new log.
I20260809 03:37:51.068785 24767 tablet_bootstrap.cc:492] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: No bootstrap required, opened a new log
I20260809 03:37:51.069027 24767 ts_tablet_manager.cc:1397] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.004s
I20260809 03:37:51.070103 24771 log.cc:824] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:51.072168 24771 tablet_bootstrap.cc:492] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800: No bootstrap required, opened a new log
I20260809 03:37:51.072388 24771 ts_tablet_manager.cc:1397] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260809 03:37:51.072700 24767 raft_consensus.cc:348] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:51.072981 24767 raft_consensus.cc:374] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:51.073065 24767 raft_consensus.cc:729] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06660db80de49d5b6b1ea3c9027d222, State: Initialized, Role: FOLLOWER
I20260809 03:37:51.073498 24767 consensus_queue.cc:260] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:51.074246 24563 heartbeater.cc:503] Master 127.1.224.126:37957 was elected leader, sending a full tablet report...
I20260809 03:37:51.074527 24767 ts_tablet_manager.cc:1428] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:37:51.076050 24771 raft_consensus.cc:348] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:51.076370 24771 raft_consensus.cc:374] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:37:51.076452 24771 raft_consensus.cc:729] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351801e790714f879833f6f5f719b800, State: Initialized, Role: FOLLOWER
I20260809 03:37:51.076911 24771 consensus_queue.cc:260] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:51.077558 24745 heartbeater.cc:503] Master 127.1.224.126:37957 was elected leader, sending a full tablet report...
I20260809 03:37:51.077984 24771 ts_tablet_manager.cc:1428] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260809 03:37:51.097227 24375 tablet.cc:2369] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:37:51.234273 24773 raft_consensus.cc:482] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:37:51.234584 24773 raft_consensus.cc:504] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:51.235917 24773 leader_election.cc:288] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 351801e790714f879833f6f5f719b800 (127.1.224.67:44013), d331db6119b043ee965a0e3e8ece4e8e (127.1.224.65:37581)
I20260809 03:37:51.239701 24566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.236134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:44013 (local address 127.1.224.66:37831)
0809 03:37:51.236304 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:51.236323 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:51.236432 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:51.236798 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:51.236802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:51.236860 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:51.236975 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:51.236980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.237743 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.237748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:51.238545 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:51.238555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.239240 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.239243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:51.239361 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.239380 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 03:37:51.239480 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 03:37:51.239559 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"spinlock_wait_cycles":22144}
I20260809 03:37:51.239701 24764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.236381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:37831 (local address 127.1.224.67:44013)
0809 03:37:51.236550 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:51.236556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:51.236571 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:37:51.236630 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:51.236632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:51.236674 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:51.236759 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:51.237098 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.237617 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.238683 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.239104 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.239241 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.239300 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:51.239497 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 03:37:51.239571 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:37:51.239984 24775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.236239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:37581 (local address 127.1.224.66:40579)
0809 03:37:51.236688 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:51.236700 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:51.236869 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:51.237158 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:51.237160 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:51.237177 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:37:51.237303 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:51.237308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.238043 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.238047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:51.238830 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:51.238838 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.239503 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.239507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:51.239632 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.239650 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 03:37:51.239740 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 03:37:51.239817 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"spinlock_wait_cycles":44672,"thread_start_us":118,"threads_started":1}
I20260809 03:37:51.240453 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.236344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:40579 (local address 127.1.224.65:37581)
0809 03:37:51.236514 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:51.236520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:51.236799 (+ 279us) server_negotiation.cc:407] Connection header received
0809 03:37:51.236994 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:51.236997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:51.237045 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:51.237145 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:37:51.237427 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.237917 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.238969 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.239371 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.239509 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.239566 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:51.239759 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 03:37:51.240338 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:37:51.240605 24674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5909470c8c68403aa38cb33505951124" candidate_uuid: "f06660db80de49d5b6b1ea3c9027d222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351801e790714f879833f6f5f719b800" is_pre_election: true
I20260809 03:37:51.240994 24674 raft_consensus.cc:2454] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f06660db80de49d5b6b1ea3c9027d222 in term 0.
I20260809 03:37:51.241101 24312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5909470c8c68403aa38cb33505951124" candidate_uuid: "f06660db80de49d5b6b1ea3c9027d222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d331db6119b043ee965a0e3e8ece4e8e" is_pre_election: true
I20260809 03:37:51.241436 24312 raft_consensus.cc:2454] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f06660db80de49d5b6b1ea3c9027d222 in term 0.
I20260809 03:37:51.241530 24403 leader_election.cc:302] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 351801e790714f879833f6f5f719b800, f06660db80de49d5b6b1ea3c9027d222; no voters:
I20260809 03:37:51.241899 24773 raft_consensus.cc:2781] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:37:51.242005 24773 raft_consensus.cc:482] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:37:51.242053 24773 raft_consensus.cc:3037] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:51.243155 24773 raft_consensus.cc:504] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:51.243507 24773 leader_election.cc:288] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [CANDIDATE]: Term 1 election: Requested vote from peers 351801e790714f879833f6f5f719b800 (127.1.224.67:44013), d331db6119b043ee965a0e3e8ece4e8e (127.1.224.65:37581)
I20260809 03:37:51.243880 24312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5909470c8c68403aa38cb33505951124" candidate_uuid: "f06660db80de49d5b6b1ea3c9027d222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d331db6119b043ee965a0e3e8ece4e8e"
I20260809 03:37:51.243875 24674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5909470c8c68403aa38cb33505951124" candidate_uuid: "f06660db80de49d5b6b1ea3c9027d222" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351801e790714f879833f6f5f719b800"
I20260809 03:37:51.244009 24674 raft_consensus.cc:3037] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:37:51.244009 24312 raft_consensus.cc:3037] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 0 FOLLOWER]: Advancing to term 1
W20260809 03:37:51.245149 24746 tablet.cc:2369] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:37:51.245374 24312 raft_consensus.cc:2454] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f06660db80de49d5b6b1ea3c9027d222 in term 1.
I20260809 03:37:51.245374 24674 raft_consensus.cc:2454] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f06660db80de49d5b6b1ea3c9027d222 in term 1.
I20260809 03:37:51.245711 24408 leader_election.cc:302] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d331db6119b043ee965a0e3e8ece4e8e, f06660db80de49d5b6b1ea3c9027d222; no voters:
I20260809 03:37:51.245893 24773 raft_consensus.cc:2781] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:37:51.246124 24773 raft_consensus.cc:686] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 1 LEADER]: Becoming Leader. State: Replica: f06660db80de49d5b6b1ea3c9027d222, State: Running, Role: LEADER
I20260809 03:37:51.246490 24773 consensus_queue.cc:237] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:51.250020 24116 catalog_manager.cc:5495] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 reported cstate change: term changed from 0 to 1, leader changed from <none> to f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66). New cstate: current_term: 1 leader_uuid: "f06660db80de49d5b6b1ea3c9027d222" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } health_report { overall_health: UNKNOWN } } }
I20260809 03:37:51.286618 24758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.282766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51008 (local address 127.1.224.66:42117)
0809 03:37:51.282932 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:51.282937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:51.282955 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:37:51.283067 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:51.283069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:51.283133 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:51.283259 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:37:51.283518 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.284057 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.285025 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:51.285255 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:51.285388 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:51.285724 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:37:51.286191 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:37:51.286337 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:51.286400 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 03:37:51.286463 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 03:37:51.293273 24312 raft_consensus.cc:1261] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Refusing update from remote peer f06660db80de49d5b6b1ea3c9027d222: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:37:51.293275 24674 raft_consensus.cc:1261] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800 [term 1 FOLLOWER]: Refusing update from remote peer f06660db80de49d5b6b1ea3c9027d222: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:37:51.294174 24776 consensus_queue.cc:1035] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:37:51.294384 24773 consensus_queue.cc:1035] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:37:51.294577 24565 tablet.cc:2369] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:37:51.302727 24780 mvcc.cc:205] Tried to move back new op lower bound from 7316466365608902656 to 7316466365428457472. Current Snapshot: MvccSnapshot[applied={T|T < 7316466365608902656}]
I20260809 03:37:51.311970 24116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:37:51.312171 24116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:37:51.314644 24116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Sending DeleteTablet for 3 replicas of tablet 5909470c8c68403aa38cb33505951124
I20260809 03:37:51.315573 24470 tablet_service.cc:1514] Processing DeleteTablet for tablet 5909470c8c68403aa38cb33505951124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:37:51 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:50996
I20260809 03:37:51.315652 24279 tablet_service.cc:1514] Processing DeleteTablet for tablet 5909470c8c68403aa38cb33505951124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:37:51 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:58766
I20260809 03:37:51.315778 24649 tablet_service.cc:1514] Processing DeleteTablet for tablet 5909470c8c68403aa38cb33505951124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:37:51 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:34630
I20260809 03:37:51.316381 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 24168
I20260809 03:37:51.316419 24793 tablet_replica.cc:330] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: stopping tablet replica
I20260809 03:37:51.329516 24792 tablet_replica.cc:330] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: stopping tablet replica
I20260809 03:37:51.330088 24792 raft_consensus.cc:2229] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:37:51.330701 24792 raft_consensus.cc:2258] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:37:51.316406 24794 tablet_replica.cc:330] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800: stopping tablet replica
I20260809 03:37:51.333462 24794 raft_consensus.cc:2229] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:37:51.333969 24794 raft_consensus.cc:2258] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:37:51.334102 24792 ts_tablet_manager.cc:1905] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:37:51.335939 24794 ts_tablet_manager.cc:1905] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:37:51.337986 24792 ts_tablet_manager.cc:1918] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:37:51.338132 24792 log.cc:1196] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/5909470c8c68403aa38cb33505951124
W20260809 03:37:51.338378 24101 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:37581 (error 108)
I20260809 03:37:51.338764 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 24376
I20260809 03:37:51.349046 24794 ts_tablet_manager.cc:1918] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:37:51.349192 24794 log.cc:1196] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/5909470c8c68403aa38cb33505951124
I20260809 03:37:51.349573 24794 ts_tablet_manager.cc:1939] T 5909470c8c68403aa38cb33505951124 P 351801e790714f879833f6f5f719b800: Deleting consensus metadata
I20260809 03:37:51.351418 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.351042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:37581 (local address 127.0.0.1:58782)
0809 03:37:51.351209 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:51.351307 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:37581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 03:37:51.351469 24098 catalog_manager.cc:4841] TS 351801e790714f879833f6f5f719b800 (127.1.224.67:44013): tablet 5909470c8c68403aa38cb33505951124 (table test-table [id=7f8cef9aed1340779285d66a1b6f615b]) successfully deleted
W20260809 03:37:51.351732 24101 catalog_manager.cc:4584] TS d331db6119b043ee965a0e3e8ece4e8e (127.1.224.65:37581): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5909470c8c68403aa38cb33505951124: Network error: Client connection negotiation failed: client connection to 127.1.224.65:37581: connect: Connection refused (error 111)
I20260809 03:37:51.358049 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 24568
I20260809 03:37:51.363683 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.358175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:42117 (local address 127.0.0.1:51012)
0809 03:37:51.363458 (+ 5283us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:51.363550 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.66:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5118}
I20260809 03:37:51.373692 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 24073
I20260809 03:37:51.395017 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37957
--webserver_interface=127.1.224.126
--webserver_port=39549
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:37957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:37:51.514868 24797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:51.515177 24797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:51.515219 24797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:51.519479 24797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:37:51.519559 24797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:51.519579 24797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:51.519596 24797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:37:51.519613 24797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:37:51.525533 24797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:37957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=39549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:51.526791 24797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:51.527940 24797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:51.537077 24806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:51.537132 24804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:51.537127 24808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:51.537444 24797 server_base.cc:1034] running on GCE node
I20260809 03:37:51.538044 24797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:51.539247 24797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:51.540439 24797 hybrid_clock.cc:648] HybridClock initialized: now 1786246671540423 us; error 28 us; skew 500 ppm
Aug 09 03:37:51 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246671, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:51.543352 24797 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:37:51.544281 24797 webserver.cc:460] Webserver started at http://127.1.224.126:39549/ using document root <none> and password file <none>
I20260809 03:37:51.544939 24797 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:51.545034 24797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:51.548974 24797 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:37:51.551283 24819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:51.552650 24797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:37:51.552798 24797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "5ae42346c98041c6a0ed665b75da1847"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "9406594f6267e54448731ab092975f62"
server_key_iv: "4a7a5de7c70b8cd4c6d889c4e03140a8"
server_key_version: "encryptionkey@0"
I20260809 03:37:51.553421 24797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:51.572122 24797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:51.575542 24797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:51.575803 24797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:51.583897 24797 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:37957
I20260809 03:37:51.584245 24887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:37957 every 8 connection(s)
I20260809 03:37:51.585175 24797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:37:51.590056 24888 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:37:51.591946 24888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Bootstrap starting.
I20260809 03:37:51.593227 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 24797
I20260809 03:37:51.593681 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:37581
--local_ip_for_outbound_sockets=127.1.224.65
--tserver_master_addrs=127.1.224.126:37957
--webserver_port=33811
--webserver_interface=127.1.224.65
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:37:51.617036 24888 log.cc:824] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:51.632376 24888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:37:51.632953 24888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847: Bootstrap complete.
I20260809 03:37:51.638723 24888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } }
I20260809 03:37:51.639343 24888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae42346c98041c6a0ed665b75da1847, State: Initialized, Role: FOLLOWER
I20260809 03:37:51.640015 24888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } }
I20260809 03:37:51.640209 24888 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:37:51.640290 24888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:37:51.640396 24888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 2 FOLLOWER]: Advancing to term 3
I20260809 03:37:51.643240 24888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } }
I20260809 03:37:51.643644 24888 leader_election.cc:302] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ae42346c98041c6a0ed665b75da1847; no voters:
I20260809 03:37:51.644073 24888 leader_election.cc:288] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 03:37:51.644250 24894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 03:37:51.644570 24894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [term 3 LEADER]: Becoming Leader. State: Replica: 5ae42346c98041c6a0ed665b75da1847, State: Running, Role: LEADER
I20260809 03:37:51.644948 24894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } }
I20260809 03:37:51.645512 24888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:37:51.646612 24895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5ae42346c98041c6a0ed665b75da1847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } } }
I20260809 03:37:51.646641 24898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ae42346c98041c6a0ed665b75da1847. Latest consensus state: current_term: 3 leader_uuid: "5ae42346c98041c6a0ed665b75da1847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ae42346c98041c6a0ed665b75da1847" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37957 } } }
I20260809 03:37:51.646772 24898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:51.646772 24895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ae42346c98041c6a0ed665b75da1847 [sys.catalog]: This master's current role is: LEADER
I20260809 03:37:51.647372 24909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:37:51.653128 24909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:37:51.653817 24909 catalog_manager.cc:1259] Loaded cluster ID: 479838f86abf4920a3963a37dad7bb18
I20260809 03:37:51.653883 24909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 03:37:51.655045 24909 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f744a9da9ad google::LogMessage::Fail() at ??:0
@ 0x7f744a9dd88a google::LogMessage::SendToLog() at ??:0
@ 0x7f744a9da4de google::LogMessage::Flush() at ??:0
@ 0x7f744a9dba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f745531dfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f745531ed2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f745531b579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f745535f640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f744cc2c3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f744cc2cc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f744cc2e3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f744cc1d512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f744e2616db start_thread at ??:0
@ 0x7f74494d861f clone at ??:0
W20260809 03:37:51.772778 24892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:51.773216 24892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:51.773298 24892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:51.779634 24892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:51.779736 24892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:51.779834 24892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:37:51.788857 24892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:37581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=33811
--tserver_master_addrs=127.1.224.126:37957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:51.790417 24892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:51.791996 24892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:51.805167 24952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:51.805253 24953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:51.805553 24892 server_base.cc:1034] running on GCE node
W20260809 03:37:51.805281 24958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:51.806084 24892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:51.806864 24892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:51.808108 24892 hybrid_clock.cc:648] HybridClock initialized: now 1786246671808072 us; error 45 us; skew 500 ppm
Aug 09 03:37:51 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246671, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:51.811652 24892 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:37:51.812855 24892 webserver.cc:460] Webserver started at http://127.1.224.65:33811/ using document root <none> and password file <none>
I20260809 03:37:51.813514 24892 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:51.813617 24892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:51.818018 24892 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:37:51.820775 24970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:51.822177 24892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:37:51.822280 24892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "d331db6119b043ee965a0e3e8ece4e8e"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "00bce3f9e39861b9817b9dd2b06a90b5"
server_key_iv: "665f3a907942df32e23d7048aa6d85ed"
server_key_version: "encryptionkey@0"
I20260809 03:37:51.822887 24892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:51.844720 24892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:51.848371 24892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:51.848631 24892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:51.849339 24892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:51.850589 24979 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:37:51.856642 24892 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:37:51.856743 24892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260809 03:37:51.856868 24892 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:37:51.858702 24892 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:37:51.858781 24892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 03:37:51.859127 24979 tablet_bootstrap.cc:492] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: Bootstrap starting.
I20260809 03:37:51.870083 24892 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:37581
I20260809 03:37:51.870338 25130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:37581 every 8 connection(s)
I20260809 03:37:51.871232 24892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:37:51.873788 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 24892
I20260809 03:37:51.874284 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:42117
--local_ip_for_outbound_sockets=127.1.224.66
--tserver_master_addrs=127.1.224.126:37957
--webserver_port=43699
--webserver_interface=127.1.224.66
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:37:51.881814 24979 log.cc:824] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: Log is configured to *not* fsync() on all Append() calls
I20260809 03:37:51.892231 24979 tablet_bootstrap.cc:492] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:37:51.892788 24979 tablet_bootstrap.cc:492] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: Bootstrap complete.
I20260809 03:37:51.894363 24979 ts_tablet_manager.cc:1397] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: Time spent bootstrapping tablet: real 0.036s user 0.026s sys 0.007s
I20260809 03:37:51.897994 24979 raft_consensus.cc:348] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:51.898627 24979 raft_consensus.cc:729] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d331db6119b043ee965a0e3e8ece4e8e, State: Initialized, Role: FOLLOWER
I20260809 03:37:51.899288 24979 consensus_queue.cc:260] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:51.900588 24979 ts_tablet_manager.cc:1428] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
W20260809 03:37:51.917562 25135 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:37:51.873536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:45683)
0809 03:37:51.874877 (+ 1341us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:51.874943 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:51.876228 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:51.916663 (+ 40435us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1068,"thread_start_us":184,"threads_started":1}
W20260809 03:37:51.918344 25131 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:37957 (0 consecutive failures): Network error: Failed to ping master at 127.1.224.126:37957: Client connection negotiation failed: client connection to 127.1.224.126:37957: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:37:51.919238 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.918948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:38083)
0809 03:37:51.919099 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:51.919170 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 03:37:51.920023 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:51.919769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:37833)
0809 03:37:51.919923 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:51.919969 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 03:37:51.920202 25131 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:37:52.002040 25136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:52.002333 25136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:52.002414 25136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:52.006532 25136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:52.006623 25136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:52.006716 25136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:37:52.012149 25136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:42117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=43699
--tserver_master_addrs=127.1.224.126:37957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:52.013445 25136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:52.014520 25136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:52.024681 25146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:52.024698 25151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:52.024712 25148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:52.024982 25136 server_base.cc:1034] running on GCE node
I20260809 03:37:52.025405 25136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:52.026129 25136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:52.027290 25136 hybrid_clock.cc:648] HybridClock initialized: now 1786246672027260 us; error 42 us; skew 500 ppm
Aug 09 03:37:52 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246672, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:52.030084 25136 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:37:52.031020 25136 webserver.cc:460] Webserver started at http://127.1.224.66:43699/ using document root <none> and password file <none>
I20260809 03:37:52.031579 25136 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:52.031642 25136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:52.035898 25136 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:37:52.038266 25161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:52.039592 25136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:37:52.039736 25136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "f06660db80de49d5b6b1ea3c9027d222"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "6c051a67f43edd4d007318acc4eb13fa"
server_key_iv: "1e3059737f0a13c42f8b7973b36810f3"
server_key_version: "encryptionkey@0"
I20260809 03:37:52.040343 25136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:52.055208 25136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:52.058950 25136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:52.059206 25136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:52.059894 25136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:52.061105 25170 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260809 03:37:52.065294 25170 ts_tablet_manager.cc:1852] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260809 03:37:52.065726 25170 ts_tablet_manager.cc:1905] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:37:52.069770 25170 ts_tablet_manager.cc:1918] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:37:52.069929 25170 log.cc:1196] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/5909470c8c68403aa38cb33505951124
I20260809 03:37:52.070338 25170 ts_tablet_manager.cc:1939] T 5909470c8c68403aa38cb33505951124 P f06660db80de49d5b6b1ea3c9027d222: Deleting consensus metadata
I20260809 03:37:52.070696 25136 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260809 03:37:52.070760 25136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.001s
I20260809 03:37:52.070858 25136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:52.070897 25136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:52.081005 25136 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:42117
I20260809 03:37:52.081048 25326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:42117 every 8 connection(s)
I20260809 03:37:52.082036 25136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:37:52.084762 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 25136
I20260809 03:37:52.085270 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:44013
--local_ip_for_outbound_sockets=127.1.224.67
--tserver_master_addrs=127.1.224.126:37957
--webserver_port=36597
--webserver_interface=127.1.224.67
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:37:52.086169 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:52.084571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:37333)
0809 03:37:52.085192 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:52.085311 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":174,"threads_started":1}
W20260809 03:37:52.086776 25327 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:37957 (0 consecutive failures): Network error: Failed to ping master at 127.1.224.126:37957: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
I20260809 03:37:52.087581 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:52.087334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:33933)
0809 03:37:52.087467 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:52.087520 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 03:37:52.088274 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:52.088056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:35615)
0809 03:37:52.088171 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:52.088223 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 03:37:52.088425 25327 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:37:52.204643 25332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:37:52.204974 25332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:37:52.205066 25332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:37:52.209277 25332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:37:52.209378 25332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:37:52.209475 25332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:37:52.215193 25332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:44013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=36597
--tserver_master_addrs=127.1.224.126:37957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:37:52.216400 25332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:37:52.217531 25332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:37:52.227789 25343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:52.227972 25332 server_base.cc:1034] running on GCE node
W20260809 03:37:52.227782 25341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:37:52.227782 25340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:37:52.228631 25332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:37:52.229332 25332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:37:52.230530 25332 hybrid_clock.cc:648] HybridClock initialized: now 1786246672230492 us; error 50 us; skew 500 ppm
Aug 09 03:37:52 dist-test-slave-jq4t krb5kdc[23327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246672, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:37:52.233580 25332 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:37:52.234643 25332 webserver.cc:460] Webserver started at http://127.1.224.67:36597/ using document root <none> and password file <none>
I20260809 03:37:52.235280 25332 fs_manager.cc:359] Metadata directory not provided
I20260809 03:37:52.235384 25332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:37:52.239631 25332 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:37:52.241935 25353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:52.243186 25332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:37:52.243347 25332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "351801e790714f879833f6f5f719b800"
format_stamp: "Formatted at 2026-08-09 03:37:49 on dist-test-slave-jq4t"
server_key: "7cc5aab7f18d16c0577dd51772c60e47"
server_key_iv: "df23e315ca7a151dcbe1c80c1b97f7a4"
server_key_version: "encryptionkey@0"
I20260809 03:37:52.243985 25332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:37:52.259845 25332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:37:52.263000 25332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:37:52.263233 25332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:37:52.263906 25332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:37:52.264853 25332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:37:52.264930 25332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:52.265007 25332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:37:52.265053 25332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:37:52.274549 25332 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:44013
I20260809 03:37:52.274569 25523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:44013 every 8 connection(s)
I20260809 03:37:52.275655 25332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:37:52.279464 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:52.277908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:51721)
0809 03:37:52.278597 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:52.278710 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":114,"threads_started":1}
W20260809 03:37:52.279986 25524 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:37957 (0 consecutive failures): Network error: Failed to ping master at 127.1.224.126:37957: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
I20260809 03:37:52.280689 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:52.280417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:50819)
0809 03:37:52.280580 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:52.280630 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 03:37:52.281327 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:52.281133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:40237)
0809 03:37:52.281235 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:52.281277 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 03:37:52.281544 25524 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 03:37:52.283931 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 25332
I20260809 03:37:52.921363 25530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:52.920869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:38055)
0809 03:37:52.921179 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:52.921237 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":80,"threads_started":1}
I20260809 03:37:53.085211 25531 raft_consensus.cc:482] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:37:53.085430 25531 raft_consensus.cc:504] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:53.086532 25531 leader_election.cc:288] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117), 351801e790714f879833f6f5f719b800 (127.1.224.67:44013)
I20260809 03:37:53.089478 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:53.088984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:53365)
0809 03:37:53.089310 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:53.089367 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":84,"threads_started":1}
Aug 09 03:37:53 dist-test-slave-jq4t krb5kdc[23327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246671, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.67@KRBTEST.COM
Aug 09 03:37:53 dist-test-slave-jq4t krb5kdc[23327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246671, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.66@KRBTEST.COM
I20260809 03:37:53.101145 25530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:53.086832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:42117 (local address 127.1.224.65:60203)
0809 03:37:53.086986 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:53.087001 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:53.087122 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:53.089214 (+ 2092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:53.089220 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:53.089751 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:53.090199 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:53.090214 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:53.091565 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:53.091568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:53.092467 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:53.092525 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:53.093256 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:53.093260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:53.093481 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:53.094231 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:53.094243 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:53.096175 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:53.098615 (+ 2440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:53.098619 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:53.098621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:53.098887 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:53.099317 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:53.099320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:53.099322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:53.099432 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:53.100176 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:53.100179 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:53.100297 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 03:37:53.100544 (+ 247us) client_negotiation.cc:240] Negotiation successful
0809 03:37:53.100931 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":116,"spinlock_wait_cycles":18176}
I20260809 03:37:53.101158 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:53.087150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:44013 (local address 127.1.224.65:45231)
0809 03:37:53.087419 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:53.087432 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:37:53.087556 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:37:53.089752 (+ 2196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:37:53.089755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:37:53.089942 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:37:53.090198 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:53.090214 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:53.091556 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:53.091561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:53.092462 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:37:53.092531 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:37:53.093431 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:37:53.093434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:37:53.093578 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:53.094182 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:37:53.094200 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:37:53.095790 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:37:53.098090 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:53.098097 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:53.098111 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:53.098498 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:53.098916 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:37:53.098922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:37:53.098925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:37:53.099084 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:37:53.099992 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:37:53.099998 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:37:53.100297 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 03:37:53.100544 (+ 247us) client_negotiation.cc:240] Negotiation successful
0809 03:37:53.100987 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":239,"spinlock_wait_cycles":14080,"thread_start_us":78,"threads_started":1}
I20260809 03:37:53.101806 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:53.087366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:45231 (local address 127.1.224.67:44013)
0809 03:37:53.087700 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:53.087711 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:53.087743 (+ 32us) server_negotiation.cc:407] Connection header received
0809 03:37:53.088614 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:53.088621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:53.088996 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:53.089314 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:53.090392 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:53.091412 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:53.092822 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:53.093295 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:53.093486 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:53.096054 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:53.096061 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:53.096081 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:53.096122 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:53.097956 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:53.098628 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:53.098634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:53.098640 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:53.098789 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:53.099221 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:53.099224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:53.099226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:53.099851 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:53.100000 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:53.101296 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0809 03:37:53.101604 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":97,"threads_started":1}
I20260809 03:37:53.101035 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:53.086941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:60203 (local address 127.1.224.66:42117)
0809 03:37:53.087398 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0809 03:37:53.087414 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:37:53.087459 (+ 45us) server_negotiation.cc:407] Connection header received
0809 03:37:53.088350 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:37:53.088361 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:37:53.088838 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:37:53.089148 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:37:53.090388 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:53.091397 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:53.092651 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:37:53.093126 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:37:53.093333 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:37:53.096357 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:37:53.096362 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:37:53.096372 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:37:53.096399 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:37:53.098452 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:53.099003 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:53.099009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:53.099014 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:53.099161 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:37:53.099545 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:37:53.099549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:37:53.099551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:37:53.100036 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:37:53.100164 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 03:37:53.100657 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 03:37:53.100886 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":119,"threads_started":1}
W20260809 03:37:53.104195 24971 leader_election.cc:341] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 351801e790714f879833f6f5f719b800 (127.1.224.67:44013): Not found: Tablet not found: 5909470c8c68403aa38cb33505951124
W20260809 03:37:53.104444 24971 leader_election.cc:341] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117): Not found: Tablet not found: 5909470c8c68403aa38cb33505951124
I20260809 03:37:53.104561 24971 leader_election.cc:302] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d331db6119b043ee965a0e3e8ece4e8e; no voters: 351801e790714f879833f6f5f719b800, f06660db80de49d5b6b1ea3c9027d222
I20260809 03:37:53.104871 25531 raft_consensus.cc:2726] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:37:53.282541 25539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:53.282115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:46741)
0809 03:37:53.282380 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:53.282438 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":99,"threads_started":1}
I20260809 03:37:53.923055 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:53.922569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:59041)
0809 03:37:53.922881 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:53.922946 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":85,"threads_started":1}
I20260809 03:37:54.090965 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:54.090468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:34551)
0809 03:37:54.090767 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:54.090851 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":104,"threads_started":1}
I20260809 03:37:54.283779 25542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:54.283325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:56927)
0809 03:37:54.283596 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:54.283657 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
I20260809 03:37:54.924723 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:54.924134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:48349)
0809 03:37:54.924465 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:54.924589 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":122,"threads_started":1}
I20260809 03:37:55.092633 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:55.092069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:43145)
0809 03:37:55.092384 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:55.092445 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":116,"threads_started":1}
I20260809 03:37:55.285089 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:55.284646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:38977)
0809 03:37:55.284929 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:55.284987 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":76,"threads_started":1}
I20260809 03:37:55.337579 25547 raft_consensus.cc:482] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:37:55.337724 25547 raft_consensus.cc:504] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:55.338168 25547 leader_election.cc:288] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117), 351801e790714f879833f6f5f719b800 (127.1.224.67:44013)
W20260809 03:37:55.339134 24971 leader_election.cc:341] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117): Not found: Tablet not found: 5909470c8c68403aa38cb33505951124
W20260809 03:37:55.339244 24971 leader_election.cc:341] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 351801e790714f879833f6f5f719b800 (127.1.224.67:44013): Not found: Tablet not found: 5909470c8c68403aa38cb33505951124
I20260809 03:37:55.339280 24971 leader_election.cc:302] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d331db6119b043ee965a0e3e8ece4e8e; no voters: 351801e790714f879833f6f5f719b800, f06660db80de49d5b6b1ea3c9027d222
I20260809 03:37:55.339426 25547 raft_consensus.cc:2726] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:37:55.926218 25548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:55.925649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:55585)
0809 03:37:55.925987 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:55.926074 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":111,"threads_started":1}
I20260809 03:37:56.093835 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:56.093404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:49719)
0809 03:37:56.093674 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:56.093738 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":90,"threads_started":1}
I20260809 03:37:56.286371 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:56.285924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:36737)
0809 03:37:56.286205 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:56.286266 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":84,"threads_started":1}
I20260809 03:37:56.927875 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:56.927403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:36195)
0809 03:37:56.927708 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:56.927768 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":80,"threads_started":1}
I20260809 03:37:56.969532 25553 raft_consensus.cc:482] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:37:56.969708 25553 raft_consensus.cc:504] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:37:56.970144 25553 leader_election.cc:288] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117), 351801e790714f879833f6f5f719b800 (127.1.224.67:44013)
W20260809 03:37:56.971156 24971 leader_election.cc:341] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 351801e790714f879833f6f5f719b800 (127.1.224.67:44013): Not found: Tablet not found: 5909470c8c68403aa38cb33505951124
W20260809 03:37:56.971314 24971 leader_election.cc:341] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117): Not found: Tablet not found: 5909470c8c68403aa38cb33505951124
I20260809 03:37:56.971376 24971 leader_election.cc:302] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d331db6119b043ee965a0e3e8ece4e8e; no voters: 351801e790714f879833f6f5f719b800, f06660db80de49d5b6b1ea3c9027d222
I20260809 03:37:56.971601 25553 raft_consensus.cc:2726] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:37:57.095129 25554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:57.094677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:59381)
0809 03:37:57.094968 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:57.095030 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":99,"threads_started":1}
I20260809 03:37:57.287840 25556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:57.287296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:55913)
0809 03:37:57.287672 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:57.287738 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":82,"threads_started":1}
I20260809 03:37:57.929293 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:57.928796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:42239)
0809 03:37:57.929108 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:57.929173 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":86,"threads_started":1}
I20260809 03:37:58.096539 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:58.096030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:55227)
0809 03:37:58.096310 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:58.096375 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":83,"threads_started":1}
I20260809 03:37:58.289561 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:58.289056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:48231)
0809 03:37:58.289369 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:58.289449 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":90,"threads_started":1}
I20260809 03:37:58.930748 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:58.930248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:35065)
0809 03:37:58.930566 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:58.930632 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":83,"threads_started":1}
I20260809 03:37:59.098012 25564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:59.097437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:34143)
0809 03:37:59.097832 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:59.097905 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":95,"threads_started":1}
I20260809 03:37:59.291198 25565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:59.290591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:42117)
0809 03:37:59.291017 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:59.291089 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":98,"threads_started":1}
I20260809 03:37:59.932163 25568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:37:59.931657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:57833)
0809 03:37:59.931988 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 03:37:59.932054 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":96,"threads_started":1}
I20260809 03:38:00.099470 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:00.098946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:50931)
0809 03:38:00.099256 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:00.099334 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":112,"threads_started":1}
I20260809 03:38:00.292649 25570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:00.292158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:54043)
0809 03:38:00.292419 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:00.292532 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":80,"threads_started":1}
I20260809 03:38:00.933573 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:00.933060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:60615)
0809 03:38:00.933360 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:00.933435 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":107,"threads_started":1}
I20260809 03:38:01.101121 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:01.100615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:35563)
0809 03:38:01.100942 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:01.101014 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":93,"threads_started":1}
I20260809 03:38:01.294044 25574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:01.293540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:35741)
0809 03:38:01.293871 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:01.293934 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":106,"threads_started":1}
I20260809 03:38:01.459520 25575 raft_consensus.cc:482] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:38:01.459666 25575 raft_consensus.cc:504] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:38:01.460136 25575 leader_election.cc:288] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117), 351801e790714f879833f6f5f719b800 (127.1.224.67:44013)
W20260809 03:38:01.461076 24971 leader_election.cc:341] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 351801e790714f879833f6f5f719b800 (127.1.224.67:44013): Not found: Tablet not found: 5909470c8c68403aa38cb33505951124
W20260809 03:38:01.461237 24971 leader_election.cc:341] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117): Not found: Tablet not found: 5909470c8c68403aa38cb33505951124
I20260809 03:38:01.461311 24971 leader_election.cc:302] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d331db6119b043ee965a0e3e8ece4e8e; no voters: 351801e790714f879833f6f5f719b800, f06660db80de49d5b6b1ea3c9027d222
I20260809 03:38:01.461478 25575 raft_consensus.cc:2726] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:38:01.935220 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:01.934699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:47375)
0809 03:38:01.935049 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:01.935110 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":126,"threads_started":1}
I20260809 03:38:02.102532 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:02.102014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:46413)
0809 03:38:02.102357 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:02.102426 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":81,"threads_started":1}
I20260809 03:38:02.295545 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:02.295056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:52971)
0809 03:38:02.295365 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:02.295439 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":86,"threads_started":1}
I20260809 03:38:02.936610 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:02.936124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:52923)
0809 03:38:02.936391 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:02.936446 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":82,"threads_started":1}
I20260809 03:38:03.103765 25583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:03.103316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:57657)
0809 03:38:03.103595 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:03.103658 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":91,"threads_started":1}
I20260809 03:38:03.297142 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:03.296521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:36875)
0809 03:38:03.296885 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:03.296979 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":92,"threads_started":1}
I20260809 03:38:03.937893 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:03.937460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:39821)
0809 03:38:03.937730 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:03.937788 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":84,"threads_started":1}
I20260809 03:38:04.105201 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:04.104709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:37229)
0809 03:38:04.105020 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:04.105086 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":81,"threads_started":1}
I20260809 03:38:04.298784 25589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:04.298255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:43433)
0809 03:38:04.298573 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:04.298640 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":115,"threads_started":1}
I20260809 03:38:04.939464 25590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:04.938945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:35501)
0809 03:38:04.939283 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:04.939353 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":100,"threads_started":1}
I20260809 03:38:05.106659 25591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:05.106161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:43541)
0809 03:38:05.106481 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:05.106550 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":96,"threads_started":1}
I20260809 03:38:05.300087 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:05.299663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:58543)
0809 03:38:05.299929 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:05.299985 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":80,"threads_started":1}
I20260809 03:38:05.941097 25593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:05.940587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:49537)
0809 03:38:05.940905 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:05.940973 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":88,"threads_started":1}
I20260809 03:38:06.108101 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:06.107627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:41485)
0809 03:38:06.107930 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:06.107995 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":90,"threads_started":1}
I20260809 03:38:06.301606 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:06.301086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:56893)
0809 03:38:06.301391 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:06.301483 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":109,"threads_started":1}
I20260809 03:38:06.573650 25597 raft_consensus.cc:482] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:38:06.573808 25597 raft_consensus.cc:504] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f06660db80de49d5b6b1ea3c9027d222" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42117 } } peers { permanent_uuid: "351801e790714f879833f6f5f719b800" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 44013 } } peers { permanent_uuid: "d331db6119b043ee965a0e3e8ece4e8e" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37581 } }
I20260809 03:38:06.574376 25597 leader_election.cc:288] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117), 351801e790714f879833f6f5f719b800 (127.1.224.67:44013)
W20260809 03:38:06.575341 24971 leader_election.cc:341] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 351801e790714f879833f6f5f719b800 (127.1.224.67:44013): Not found: Tablet not found: 5909470c8c68403aa38cb33505951124
W20260809 03:38:06.575524 24971 leader_election.cc:341] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f06660db80de49d5b6b1ea3c9027d222 (127.1.224.66:42117): Not found: Tablet not found: 5909470c8c68403aa38cb33505951124
I20260809 03:38:06.575591 24971 leader_election.cc:302] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d331db6119b043ee965a0e3e8ece4e8e; no voters: 351801e790714f879833f6f5f719b800, f06660db80de49d5b6b1ea3c9027d222
I20260809 03:38:06.575752 25597 raft_consensus.cc:2726] T 5909470c8c68403aa38cb33505951124 P d331db6119b043ee965a0e3e8ece4e8e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:38:06.942775 25599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:06.942273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.65:47695)
0809 03:38:06.942580 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:06.942646 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":85,"threads_started":1}
I20260809 03:38:07.109750 25600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:07.109209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.66:35619)
0809 03:38:07.109532 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:07.109619 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":99,"threads_started":1}
I20260809 03:38:07.286571 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 24892
I20260809 03:38:07.303319 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:07.302847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37957 (local address 127.1.224.67:60067)
0809 03:38:07.303148 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:07.303214 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":96,"threads_started":1}
I20260809 03:38:07.305254 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 25136
I20260809 03:38:07.318434 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 25332
2026-08-09T03:38:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19870 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:38:07 dist-test-slave-jq4t krb5kdc[25604](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:38:07 dist-test-slave-jq4t krb5kdc[25604](info): set up 2 sockets
Aug 09 03:38:07 dist-test-slave-jq4t krb5kdc[25604](info): commencing operation
krb5kdc: starting...
W20260809 03:38:08.589900 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:38:08 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:38:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:38:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:38:08.720954 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37083
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:37083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:38:08.842061 25628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:38:08.842403 25628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:38:08.842449 25628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:38:08.846735 25628 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 03:38:08.846812 25628 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 03:38:08.846827 25628 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 03:38:08.846841 25628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:38:08.846865 25628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:38:08.846884 25628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:38:08.846901 25628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:38:08.846944 25628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:38:08.852811 25628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:37083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:38:08.854085 25628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:38:08.855213 25628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:38:08.864796 25628 server_base.cc:1034] running on GCE node
W20260809 03:38:08.864815 25637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:08.864764 25635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:08.865059 25634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:08.865708 25628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:38:08.866890 25628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:38:08.868104 25628 hybrid_clock.cc:648] HybridClock initialized: now 1786246688868075 us; error 45 us; skew 500 ppm
Aug 09 03:38:08 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:38:08.874046 25628 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:38:08.875101 25628 webserver.cc:460] Webserver started at http://127.1.224.126:46707/ using document root <none> and password file <none>
I20260809 03:38:08.875732 25628 fs_manager.cc:359] Metadata directory not provided
I20260809 03:38:08.875830 25628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:38:08.876087 25628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:38:08.878336 25628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "adddaff46a1d4ce7b2e1976fea32f60e"
format_stamp: "Formatted at 2026-08-09 03:38:08 on dist-test-slave-jq4t"
server_key: "04829a2b89634c7b5eaed902167d7d87"
server_key_iv: "b0571bd693da317d284f7099b29db2cf"
server_key_version: "encryptionkey@0"
I20260809 03:38:08.878937 25628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "adddaff46a1d4ce7b2e1976fea32f60e"
format_stamp: "Formatted at 2026-08-09 03:38:08 on dist-test-slave-jq4t"
server_key: "04829a2b89634c7b5eaed902167d7d87"
server_key_iv: "b0571bd693da317d284f7099b29db2cf"
server_key_version: "encryptionkey@0"
I20260809 03:38:08.882912 25628 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 03:38:08.885516 25649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:08.886883 25628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:38:08.887050 25628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "adddaff46a1d4ce7b2e1976fea32f60e"
format_stamp: "Formatted at 2026-08-09 03:38:08 on dist-test-slave-jq4t"
server_key: "04829a2b89634c7b5eaed902167d7d87"
server_key_iv: "b0571bd693da317d284f7099b29db2cf"
server_key_version: "encryptionkey@0"
I20260809 03:38:08.887184 25628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:38:08.907214 25628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:38:08.910312 25628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:38:08.910566 25628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:38:08.918222 25628 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:37083
I20260809 03:38:08.918538 25725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:37083 every 8 connection(s)
I20260809 03:38:08.919395 25628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:38:08.922358 25729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:38:08.928208 25729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Bootstrap starting.
I20260809 03:38:08.929163 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 25628
I20260809 03:38:08.929299 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:38:08.929653 1921 external_mini_cluster.cc:1428] Setting key 2ea8b001a34966517484f3283c5757ad
I20260809 03:38:08.931524 25729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Neither blocks nor log segments found. Creating new log.
I20260809 03:38:08.932341 25729 log.cc:824] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Log is configured to *not* fsync() on all Append() calls
I20260809 03:38:08.934705 25729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: No bootstrap required, opened a new log
Aug 09 03:38:08 dist-test-slave-jq4t krb5kdc[25604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:38:08.939738 25729 raft_consensus.cc:348] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } }
I20260809 03:38:08.940094 25729 raft_consensus.cc:374] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:38:08.940153 25729 raft_consensus.cc:729] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adddaff46a1d4ce7b2e1976fea32f60e, State: Initialized, Role: FOLLOWER
I20260809 03:38:08.940814 25729 consensus_queue.cc:260] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } }
I20260809 03:38:08.941015 25729 raft_consensus.cc:388] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:38:08.941089 25729 raft_consensus.cc:482] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:38:08.941197 25729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:38:08.942536 25729 raft_consensus.cc:504] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } }
I20260809 03:38:08.942983 25729 leader_election.cc:302] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adddaff46a1d4ce7b2e1976fea32f60e; no voters:
I20260809 03:38:08.943410 25729 leader_election.cc:288] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:38:08.943560 25735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:38:08.943853 25735 raft_consensus.cc:686] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 1 LEADER]: Becoming Leader. State: Replica: adddaff46a1d4ce7b2e1976fea32f60e, State: Running, Role: LEADER
I20260809 03:38:08.944298 25735 consensus_queue.cc:237] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } }
I20260809 03:38:08.944962 25729 sys_catalog.cc:564] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:38:08.946152 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:08.930936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.1.224.126:37083)
0809 03:38:08.931386 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:08.931402 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:08.931440 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:38:08.932164 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:08.932184 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:08.932621 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:08.932998 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:08.933859 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:08.934929 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:08.936103 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:08.936686 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:08.936839 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:08.939932 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:08.939940 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:08.939960 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:08.940009 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:08.942272 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:08.942822 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:08.942829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:08.942836 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:08.943003 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:08.943439 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:08.943443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:08.943445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:08.944042 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:08.944239 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:08.944675 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 03:38:08.945031 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":153,"threads_started":1}
I20260809 03:38:08.946429 25737 sys_catalog.cc:455] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } } }
I20260809 03:38:08.946579 25737 sys_catalog.cc:458] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: This master's current role is: LEADER
I20260809 03:38:08.946630 25736 sys_catalog.cc:455] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: SysCatalogTable state changed. Reason: New leader adddaff46a1d4ce7b2e1976fea32f60e. Latest consensus state: current_term: 1 leader_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } } }
I20260809 03:38:08.946709 25736 sys_catalog.cc:458] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: This master's current role is: LEADER
I20260809 03:38:08.947551 25748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:38:08.951048 25748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:38:08.957374 25748 catalog_manager.cc:1347] Generated new cluster ID: 55025cfb94654a4d919dceab17947e78
I20260809 03:38:08.957463 25748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:38:08.976918 25748 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:38:08.977972 25748 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:38:08.982599 25748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Generated new TSK 0
I20260809 03:38:08.983348 25748 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:38:09.015363 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:37083
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:38:09.137785 25767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:38:09.138123 25767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:38:09.138202 25767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:38:09.142539 25767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:38:09.142625 25767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:38:09.142721 25767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:38:09.148566 25767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:37083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:38:09.149729 25767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:38:09.150810 25767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:38:09.160811 25774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:09.160925 25767 server_base.cc:1034] running on GCE node
W20260809 03:38:09.160822 25778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:09.160811 25776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:09.161525 25767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:38:09.162235 25767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:38:09.163455 25767 hybrid_clock.cc:648] HybridClock initialized: now 1786246689163429 us; error 43 us; skew 500 ppm
Aug 09 03:38:09 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:38:09.166392 25767 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:38:09.167276 25767 webserver.cc:460] Webserver started at http://127.1.224.65:40525/ using document root <none> and password file <none>
I20260809 03:38:09.167871 25767 fs_manager.cc:359] Metadata directory not provided
I20260809 03:38:09.167965 25767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:38:09.168207 25767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:38:09.170327 25767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "8019f869209d4a408923eca695bcf52d"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "f02b770dacf9264186ed67b566f1d205"
server_key_iv: "6cf13acbfea4300899cc60547cce9557"
server_key_version: "encryptionkey@0"
I20260809 03:38:09.170945 25767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "8019f869209d4a408923eca695bcf52d"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "f02b770dacf9264186ed67b566f1d205"
server_key_iv: "6cf13acbfea4300899cc60547cce9557"
server_key_version: "encryptionkey@0"
I20260809 03:38:09.174710 25767 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:38:09.177088 25786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:09.178314 25767 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:38:09.178467 25767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "8019f869209d4a408923eca695bcf52d"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "f02b770dacf9264186ed67b566f1d205"
server_key_iv: "6cf13acbfea4300899cc60547cce9557"
server_key_version: "encryptionkey@0"
I20260809 03:38:09.178587 25767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:38:09.204581 25767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:38:09.208174 25767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:38:09.208407 25767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:38:09.209096 25767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:38:09.210005 25767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:38:09.210078 25767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:09.210155 25767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:38:09.210206 25767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:09.219385 25767 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:44633
I20260809 03:38:09.219416 25942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:44633 every 8 connection(s)
I20260809 03:38:09.220338 25767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:38:09.224016 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 25767
I20260809 03:38:09.224179 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:38:09.224548 1921 external_mini_cluster.cc:1428] Setting key da015d2786d30c6bacc74d9f4cdbf82f
Aug 09 03:38:09 dist-test-slave-jq4t krb5kdc[25604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:38:09.235733 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.222296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:45503 (local address 127.1.224.126:37083)
0809 03:38:09.222463 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:09.222468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:09.223402 (+ 934us) server_negotiation.cc:407] Connection header received
0809 03:38:09.225107 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:09.225111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:09.225179 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:09.225262 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:09.226813 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.227349 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.228937 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.229189 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.229311 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.231885 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:09.231889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:09.231891 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:09.231933 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:09.233514 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:09.234052 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:09.234056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:09.234058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:09.234148 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:09.234509 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:09.234513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:09.234515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:09.234722 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:09.234838 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:09.235409 (+ 571us) server_negotiation.cc:299] Negotiation successful
0809 03:38:09.235549 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 03:38:09.236642 25947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.222748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:45503)
0809 03:38:09.223231 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:09.223275 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:09.224852 (+ 1577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:09.225483 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:09.225490 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:09.225915 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:38:09.226644 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.226671 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.227497 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.227504 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.228806 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.228817 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.229358 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.229362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.229545 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.230197 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:38:09.230213 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:38:09.231713 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:38:09.233648 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:09.233654 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:09.233671 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:09.233943 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:09.234247 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:09.234251 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:09.234253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:09.234405 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:09.234837 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:38:09.234846 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:38:09.235160 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 03:38:09.235362 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 03:38:09.235708 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":130,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:38:09.238098 25945 heartbeater.cc:347] Connected to a master server at 127.1.224.126:37083
I20260809 03:38:09.238392 25945 heartbeater.cc:464] Registering TS with master...
I20260809 03:38:09.239014 25945 heartbeater.cc:511] Master 127.1.224.126:37083 requested a full tablet report, sending...
I20260809 03:38:09.240734 25675 ts_manager.cc:194] Registered new tserver with Master: 8019f869209d4a408923eca695bcf52d (127.1.224.65:44633)
I20260809 03:38:09.241919 25675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:45503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:38:09.253885 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:37083
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:38:09.372225 25952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:38:09.372588 25952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:38:09.372640 25952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:38:09.376637 25952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:38:09.376693 25952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:38:09.376751 25952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:38:09.384498 25952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:37083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:38:09.386430 25952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:38:09.387573 25952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:38:09.398088 25959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:09.398092 25965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:09.398088 25960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:09.398396 25952 server_base.cc:1034] running on GCE node
I20260809 03:38:09.398881 25952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:38:09.399523 25952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:38:09.400724 25952 hybrid_clock.cc:648] HybridClock initialized: now 1786246689400690 us; error 45 us; skew 500 ppm
Aug 09 03:38:09 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:38:09.403748 25952 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:38:09.404796 25952 webserver.cc:460] Webserver started at http://127.1.224.66:44037/ using document root <none> and password file <none>
I20260809 03:38:09.405418 25952 fs_manager.cc:359] Metadata directory not provided
I20260809 03:38:09.405515 25952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:38:09.405769 25952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:38:09.407918 25952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "040c0db80f74b8f474a6c9b59b1fb553"
server_key_iv: "6740a9534066236f963dfd46d48891f4"
server_key_version: "encryptionkey@0"
I20260809 03:38:09.408612 25952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "040c0db80f74b8f474a6c9b59b1fb553"
server_key_iv: "6740a9534066236f963dfd46d48891f4"
server_key_version: "encryptionkey@0"
I20260809 03:38:09.412599 25952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:38:09.415122 25974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:09.416373 25952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:38:09.416574 25952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "040c0db80f74b8f474a6c9b59b1fb553"
server_key_iv: "6740a9534066236f963dfd46d48891f4"
server_key_version: "encryptionkey@0"
I20260809 03:38:09.416700 25952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:38:09.434017 25952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:38:09.437427 25952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:38:09.437644 25952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:38:09.438264 25952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:38:09.439148 25952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:38:09.439232 25952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:09.439327 25952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:38:09.439378 25952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:09.449409 25952 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:35615
I20260809 03:38:09.449463 26128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:35615 every 8 connection(s)
I20260809 03:38:09.450492 25952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:38:09.451637 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 25952
I20260809 03:38:09.451772 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:38:09.452119 1921 external_mini_cluster.cc:1428] Setting key 2e262792255e92de5e8ce39fb1359f79
Aug 09 03:38:09 dist-test-slave-jq4t krb5kdc[25604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:38:09.467310 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.452599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:39915 (local address 127.1.224.126:37083)
0809 03:38:09.452748 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:09.452752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:09.454372 (+ 1620us) server_negotiation.cc:407] Connection header received
0809 03:38:09.456356 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:09.456359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:09.456413 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:09.456564 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:09.457927 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.458505 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.459798 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.460078 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.460212 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.463395 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:09.463399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:09.463401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:09.463427 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:09.465015 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:09.465579 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:09.465585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:09.465588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:09.465679 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:09.466073 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:09.466076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:09.466078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:09.466291 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:09.466413 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:09.466983 (+ 570us) server_negotiation.cc:299] Negotiation successful
0809 03:38:09.467132 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
I20260809 03:38:09.468393 26131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.453118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:39915)
0809 03:38:09.454183 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:09.454248 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:09.456097 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:09.456784 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:09.456792 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:09.457212 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:38:09.457760 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.457773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.458661 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.458664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.459637 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.459648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.460271 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.460275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.460543 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.461265 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:38:09.461293 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:38:09.463195 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:38:09.465150 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:09.465157 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:09.465176 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:09.465456 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:09.465783 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:09.465786 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:09.465789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:09.465953 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:09.466425 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:38:09.466434 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:38:09.466724 (+ 290us) client_negotiation.cc:769] Sending connection context
0809 03:38:09.466936 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 03:38:09.467312 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":141,"threads_started":1}
I20260809 03:38:09.469715 26129 heartbeater.cc:347] Connected to a master server at 127.1.224.126:37083
I20260809 03:38:09.470000 26129 heartbeater.cc:464] Registering TS with master...
I20260809 03:38:09.470539 26129 heartbeater.cc:511] Master 127.1.224.126:37083 requested a full tablet report, sending...
I20260809 03:38:09.471757 25675 ts_manager.cc:194] Registered new tserver with Master: 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615)
I20260809 03:38:09.472368 25675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:39915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:38:09.485088 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:37083
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:38:09.606076 26135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:38:09.606371 26135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:38:09.606417 26135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:38:09.610405 26135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:38:09.610481 26135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:38:09.610540 26135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:38:09.616055 26135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:37083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:38:09.617260 26135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:38:09.618323 26135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:38:09.628785 26145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:09.628786 26143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:09.628829 26135 server_base.cc:1034] running on GCE node
W20260809 03:38:09.628780 26148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:09.629582 26135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:38:09.630259 26135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:38:09.631445 26135 hybrid_clock.cc:648] HybridClock initialized: now 1786246689631411 us; error 48 us; skew 500 ppm
Aug 09 03:38:09 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:38:09.634455 26135 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:38:09.635478 26135 webserver.cc:460] Webserver started at http://127.1.224.67:33453/ using document root <none> and password file <none>
I20260809 03:38:09.636116 26135 fs_manager.cc:359] Metadata directory not provided
I20260809 03:38:09.636216 26135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:38:09.636516 26135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:38:09.638752 26135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "ae95bc86b368469ab62f81dffb6973fa"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "fd2be584eabe5e8ff31bc6616c46473b"
server_key_iv: "acab3cf682fd5ec4e93237c087c89a7d"
server_key_version: "encryptionkey@0"
I20260809 03:38:09.639372 26135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "ae95bc86b368469ab62f81dffb6973fa"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "fd2be584eabe5e8ff31bc6616c46473b"
server_key_iv: "acab3cf682fd5ec4e93237c087c89a7d"
server_key_version: "encryptionkey@0"
I20260809 03:38:09.643688 26135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 03:38:09.646682 26160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:09.648106 26135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:38:09.648267 26135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "ae95bc86b368469ab62f81dffb6973fa"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "fd2be584eabe5e8ff31bc6616c46473b"
server_key_iv: "acab3cf682fd5ec4e93237c087c89a7d"
server_key_version: "encryptionkey@0"
I20260809 03:38:09.648406 26135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:38:09.666543 26135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:38:09.670383 26135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:38:09.670625 26135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:38:09.671326 26135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:38:09.672267 26135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:38:09.672348 26135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:09.672420 26135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:38:09.672464 26135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:09.681675 26135 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:45277
I20260809 03:38:09.681761 26341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:45277 every 8 connection(s)
I20260809 03:38:09.682752 26135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:38:09.683326 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 26135
I20260809 03:38:09.683452 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:38:09.683797 1921 external_mini_cluster.cc:1428] Setting key d701cfaec09474a5d931ec4b466c6d11
Aug 09 03:38:09 dist-test-slave-jq4t krb5kdc[25604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246689, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:38:09.696404 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.684862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:32889 (local address 127.1.224.126:37083)
0809 03:38:09.685000 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:09.685005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:09.686457 (+ 1452us) server_negotiation.cc:407] Connection header received
0809 03:38:09.687304 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:09.687306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:09.687356 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:09.687430 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:09.688669 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.689236 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.690353 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.690570 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.690646 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.693276 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:09.693279 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:09.693281 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:09.693306 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:09.694571 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:09.695043 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:09.695045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:09.695047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:09.695127 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:09.695420 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:09.695422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:09.695423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:09.695603 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:09.695681 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:09.696143 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 03:38:09.696252 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:38:09.697508 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.685351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:32889)
0809 03:38:09.686312 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:09.686356 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:09.687120 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:09.687593 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:09.687600 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:09.687963 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:38:09.688465 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.688538 (+ 73us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.689344 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.689347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.690246 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.690255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.690701 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.690704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.690867 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.691537 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:38:09.691554 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:38:09.693117 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:38:09.694676 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:09.694680 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:09.694693 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:09.694940 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:09.695208 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:09.695211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:09.695212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:09.695340 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:09.695691 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:38:09.695696 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:38:09.695926 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 03:38:09.696092 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 03:38:09.696393 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":573,"threads_started":1}
I20260809 03:38:09.698664 26342 heartbeater.cc:347] Connected to a master server at 127.1.224.126:37083
I20260809 03:38:09.698982 26342 heartbeater.cc:464] Registering TS with master...
I20260809 03:38:09.699534 26342 heartbeater.cc:511] Master 127.1.224.126:37083 requested a full tablet report, sending...
I20260809 03:38:09.700919 25675 ts_manager.cc:194] Registered new tserver with Master: ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277)
I20260809 03:38:09.701504 25675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:32889
I20260809 03:38:09.710273 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:38:09.719918 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.712382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37222 (local address 127.1.224.126:37083)
0809 03:38:09.712596 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:09.712602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:09.712749 (+ 147us) server_negotiation.cc:407] Connection header received
0809 03:38:09.712880 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:09.712882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:09.712926 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:09.712987 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:09.713709 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.714238 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.715053 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.715269 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.715346 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.716900 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:09.716904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:09.716906 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:09.716931 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:09.718191 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:09.718589 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:09.718592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:09.718593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:09.718669 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:09.718930 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:09.718934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:09.718935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:09.719201 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:09.719318 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:09.719527 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 03:38:09.719634 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:38:09.723107 25675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:38:09.726302 25675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:38:09.741379 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.734814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.1.224.65:44633)
0809 03:38:09.736033 (+ 1219us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:09.736050 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:09.736080 (+ 30us) server_negotiation.cc:407] Connection header received
0809 03:38:09.736162 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:09.736202 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:09.736440 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:09.736624 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:38:09.737036 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.738518 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.739714 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.740642 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.740762 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.740954 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:09.741133 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 03:38:09.741241 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1092,"thread_start_us":145,"threads_started":1}
I20260809 03:38:09.741436 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.734909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:45277 (local address 127.0.0.1:43864)
0809 03:38:09.735394 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:09.735407 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:09.735483 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:09.736823 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:09.736827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:09.736846 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:38:09.736985 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.736993 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.738147 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.738154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.739793 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.739805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.740914 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.740918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.741008 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.741030 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 03:38:09.741078 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 03:38:09.741254 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":149,"thread_start_us":119,"threads_started":1}
I20260809 03:38:09.741436 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.734670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:44633 (local address 127.0.0.1:44756)
0809 03:38:09.735224 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:09.735278 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:09.735426 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:09.736621 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:09.736629 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:09.736660 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:38:09.736867 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.736876 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.738644 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.738647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.739576 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.739586 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.740779 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.740783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.740910 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.740964 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 03:38:09.741083 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 03:38:09.741254 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":74,"thread_start_us":107,"threads_started":1}
I20260809 03:38:09.742069 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.735446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:35615 (local address 127.0.0.1:38408)
0809 03:38:09.735671 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:09.735685 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:09.735771 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:09.736711 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:09.736714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:09.736727 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:38:09.736896 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.736905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.738382 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.738385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.739793 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.739805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.740913 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.740917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.741759 (+ 842us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.741784 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 03:38:09.741878 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 03:38:09.741958 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":842,"thread_start_us":93,"threads_started":1}
I20260809 03:38:09.742179 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.735818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38408 (local address 127.1.224.66:35615)
0809 03:38:09.736252 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:09.736262 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:09.736282 (+ 20us) server_negotiation.cc:407] Connection header received
0809 03:38:09.736346 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:09.736350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:09.736554 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:09.736868 (+ 314us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:38:09.737036 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.738237 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.739953 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.740763 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.740966 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.741142 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:09.741922 (+ 780us) server_negotiation.cc:299] Negotiation successful
0809 03:38:09.742031 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":114,"threads_started":1}
I20260809 03:38:09.742753 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.735854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43864 (local address 127.1.224.67:45277)
0809 03:38:09.736346 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:09.736357 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:09.736379 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:38:09.736450 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:09.736457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:09.736661 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:09.737131 (+ 470us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:38:09.737290 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.738008 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.739973 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.740765 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.742276 (+ 1511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.742458 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:09.742512 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 03:38:09.742608 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":161,"threads_started":1}
I20260809 03:38:09.745226 26035 tablet_service.cc:1467] Processing CreateTablet for tablet 021406e6fd464b65a907e5ae93d56281 (DEFAULT_TABLE table=test-table [id=0b667aab486949f892bc8802ddc99a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:38:09.745268 26229 tablet_service.cc:1467] Processing CreateTablet for tablet 021406e6fd464b65a907e5ae93d56281 (DEFAULT_TABLE table=test-table [id=0b667aab486949f892bc8802ddc99a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:38:09.745265 25855 tablet_service.cc:1467] Processing CreateTablet for tablet 021406e6fd464b65a907e5ae93d56281 (DEFAULT_TABLE table=test-table [id=0b667aab486949f892bc8802ddc99a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:38:09.746467 25855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 021406e6fd464b65a907e5ae93d56281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:38:09.746467 26229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 021406e6fd464b65a907e5ae93d56281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:38:09.746512 26035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 021406e6fd464b65a907e5ae93d56281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:38:09.753872 26379 tablet_bootstrap.cc:492] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Bootstrap starting.
I20260809 03:38:09.755213 26380 tablet_bootstrap.cc:492] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Bootstrap starting.
I20260809 03:38:09.756795 26379 tablet_bootstrap.cc:654] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Neither blocks nor log segments found. Creating new log.
I20260809 03:38:09.756863 26381 tablet_bootstrap.cc:492] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa: Bootstrap starting.
I20260809 03:38:09.757226 26380 tablet_bootstrap.cc:654] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Neither blocks nor log segments found. Creating new log.
I20260809 03:38:09.758133 26380 log.cc:824] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:38:09.758141 26379 log.cc:824] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:38:09.759027 26381 tablet_bootstrap.cc:654] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa: Neither blocks nor log segments found. Creating new log.
I20260809 03:38:09.760263 26379 tablet_bootstrap.cc:492] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: No bootstrap required, opened a new log
I20260809 03:38:09.760311 26381 log.cc:824] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa: Log is configured to *not* fsync() on all Append() calls
I20260809 03:38:09.760530 26379 ts_tablet_manager.cc:1397] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:38:09.760831 26380 tablet_bootstrap.cc:492] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: No bootstrap required, opened a new log
I20260809 03:38:09.761060 26380 ts_tablet_manager.cc:1397] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260809 03:38:09.762444 26381 tablet_bootstrap.cc:492] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa: No bootstrap required, opened a new log
I20260809 03:38:09.762656 26381 ts_tablet_manager.cc:1397] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:38:09.763983 26379 raft_consensus.cc:348] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } } peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } }
I20260809 03:38:09.764259 26379 raft_consensus.cc:374] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:38:09.764338 26379 raft_consensus.cc:729] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c3d6d5fdfe248e3b09b68dc8e8fdd5f, State: Initialized, Role: FOLLOWER
I20260809 03:38:09.764932 26379 consensus_queue.cc:260] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } } peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } }
I20260809 03:38:09.765663 26380 raft_consensus.cc:348] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } } peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } }
I20260809 03:38:09.765957 26380 raft_consensus.cc:374] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:38:09.766069 26380 raft_consensus.cc:729] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8019f869209d4a408923eca695bcf52d, State: Initialized, Role: FOLLOWER
I20260809 03:38:09.766080 26379 ts_tablet_manager.cc:1428] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:38:09.766491 26129 heartbeater.cc:503] Master 127.1.224.126:37083 was elected leader, sending a full tablet report...
I20260809 03:38:09.766552 26380 consensus_queue.cc:260] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } } peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } }
I20260809 03:38:09.766669 26381 raft_consensus.cc:348] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } } peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } }
I20260809 03:38:09.766948 26381 raft_consensus.cc:374] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:38:09.767011 26381 raft_consensus.cc:729] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae95bc86b368469ab62f81dffb6973fa, State: Initialized, Role: FOLLOWER
I20260809 03:38:09.767444 26381 consensus_queue.cc:260] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } } peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } }
I20260809 03:38:09.767659 26380 ts_tablet_manager.cc:1428] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260809 03:38:09.768360 26342 heartbeater.cc:503] Master 127.1.224.126:37083 was elected leader, sending a full tablet report...
I20260809 03:38:09.768412 26381 ts_tablet_manager.cc:1428] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 03:38:09.768676 25945 heartbeater.cc:503] Master 127.1.224.126:37083 was elected leader, sending a full tablet report...
I20260809 03:38:09.772748 26388 raft_consensus.cc:482] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:38:09.772918 26388 raft_consensus.cc:504] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } } peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } }
I20260809 03:38:09.774260 26388 leader_election.cc:288] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8019f869209d4a408923eca695bcf52d (127.1.224.65:44633), 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615)
I20260809 03:38:09.778069 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.774480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:44633 (local address 127.1.224.67:35131)
0809 03:38:09.774649 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:09.774667 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:09.774761 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:09.775170 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:09.775173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:09.775189 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:38:09.775285 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.775289 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.776064 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.776071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.776919 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.776929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.777624 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.777627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.777745 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.777764 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 03:38:09.777854 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 03:38:09.777931 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 03:38:09.778069 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.774566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:35131 (local address 127.1.224.65:44633)
0809 03:38:09.774712 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:09.774717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:09.774732 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:38:09.774882 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:09.774888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:09.774953 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:09.775038 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:38:09.775404 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.775933 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.777054 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.777500 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.777634 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.777694 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:09.777873 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 03:38:09.777949 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:38:09.778323 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.774881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:59859 (local address 127.1.224.66:35615)
0809 03:38:09.774998 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:09.775003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:09.775194 (+ 191us) server_negotiation.cc:407] Connection header received
0809 03:38:09.775340 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:09.775343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:09.775395 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:09.775478 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:38:09.775747 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.776246 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.777348 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.777771 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.777904 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.777955 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:09.778144 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 03:38:09.778217 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 03:38:09.778345 26389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.774782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:35615 (local address 127.1.224.67:59859)
0809 03:38:09.775100 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:09.775111 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:09.775241 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:09.775486 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:09.775489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:09.775505 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:38:09.775631 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.775643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.776358 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.776363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.777216 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:09.777226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.777897 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.777901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:09.777999 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.778029 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 03:38:09.778126 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 03:38:09.778196 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"spinlock_wait_cycles":896,"thread_start_us":143,"threads_started":1}
I20260809 03:38:09.779166 25881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "021406e6fd464b65a907e5ae93d56281" candidate_uuid: "ae95bc86b368469ab62f81dffb6973fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8019f869209d4a408923eca695bcf52d" is_pre_election: true
I20260809 03:38:09.779166 26061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "021406e6fd464b65a907e5ae93d56281" candidate_uuid: "ae95bc86b368469ab62f81dffb6973fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" is_pre_election: true
I20260809 03:38:09.779516 26061 raft_consensus.cc:2454] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae95bc86b368469ab62f81dffb6973fa in term 0.
I20260809 03:38:09.779516 25881 raft_consensus.cc:2454] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae95bc86b368469ab62f81dffb6973fa in term 0.
I20260809 03:38:09.780019 26165 leader_election.cc:302] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c3d6d5fdfe248e3b09b68dc8e8fdd5f, ae95bc86b368469ab62f81dffb6973fa; no voters:
I20260809 03:38:09.780263 26388 raft_consensus.cc:2781] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:38:09.780368 26388 raft_consensus.cc:482] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:38:09.780427 26388 raft_consensus.cc:3037] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:38:09.781582 26388 raft_consensus.cc:504] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } } peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } }
I20260809 03:38:09.781970 26388 leader_election.cc:288] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [CANDIDATE]: Term 1 election: Requested vote from peers 8019f869209d4a408923eca695bcf52d (127.1.224.65:44633), 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615)
I20260809 03:38:09.782450 25881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "021406e6fd464b65a907e5ae93d56281" candidate_uuid: "ae95bc86b368469ab62f81dffb6973fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8019f869209d4a408923eca695bcf52d"
I20260809 03:38:09.782466 26061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "021406e6fd464b65a907e5ae93d56281" candidate_uuid: "ae95bc86b368469ab62f81dffb6973fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f"
I20260809 03:38:09.782568 25881 raft_consensus.cc:3037] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:38:09.782603 26061 raft_consensus.cc:3037] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:38:09.783895 26061 raft_consensus.cc:2454] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae95bc86b368469ab62f81dffb6973fa in term 1.
I20260809 03:38:09.783913 25881 raft_consensus.cc:2454] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae95bc86b368469ab62f81dffb6973fa in term 1.
I20260809 03:38:09.784257 26165 leader_election.cc:302] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c3d6d5fdfe248e3b09b68dc8e8fdd5f, ae95bc86b368469ab62f81dffb6973fa; no voters:
I20260809 03:38:09.784451 26388 raft_consensus.cc:2781] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:38:09.784770 26388 raft_consensus.cc:686] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 1 LEADER]: Becoming Leader. State: Replica: ae95bc86b368469ab62f81dffb6973fa, State: Running, Role: LEADER
I20260809 03:38:09.785187 26388 consensus_queue.cc:237] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } } peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } }
I20260809 03:38:09.788725 25674 catalog_manager.cc:5495] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa reported cstate change: term changed from 0 to 1, leader changed from <none> to ae95bc86b368469ab62f81dffb6973fa (127.1.224.67). New cstate: current_term: 1 leader_uuid: "ae95bc86b368469ab62f81dffb6973fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260809 03:38:09.806805 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.802952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43880 (local address 127.1.224.67:45277)
0809 03:38:09.803104 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:09.803110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:09.803128 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:38:09.803235 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:09.803237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:09.803295 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:09.803419 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:38:09.803679 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.804211 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.805201 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:09.805423 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:09.805541 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:09.805878 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:38:09.806347 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:38:09.806486 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:09.806559 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 03:38:09.806623 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:38:09.813889 25881 raft_consensus.cc:1261] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Refusing update from remote peer ae95bc86b368469ab62f81dffb6973fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:38:09.813884 26061 raft_consensus.cc:1261] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 1 FOLLOWER]: Refusing update from remote peer ae95bc86b368469ab62f81dffb6973fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:38:09.814720 26388 consensus_queue.cc:1035] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:38:09.814970 26391 consensus_queue.cc:1035] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:38:09.823449 26399 mvcc.cc:205] Tried to move back new op lower bound from 7316466441469001728 to 7316466441363234816. Current Snapshot: MvccSnapshot[applied={T|T < 7316466441469001728}]
I20260809 03:38:09.833472 25674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:38:09.833690 25674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:38:09.836688 25674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Sending DeleteTablet for 3 replicas of tablet 021406e6fd464b65a907e5ae93d56281
I20260809 03:38:09.837646 26229 tablet_service.cc:1514] Processing DeleteTablet for tablet 021406e6fd464b65a907e5ae93d56281 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:38:09 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:43864
I20260809 03:38:09.837647 25855 tablet_service.cc:1514] Processing DeleteTablet for tablet 021406e6fd464b65a907e5ae93d56281 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:38:09 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:44756
I20260809 03:38:09.837791 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 25767
I20260809 03:38:09.837769 26035 tablet_service.cc:1514] Processing DeleteTablet for tablet 021406e6fd464b65a907e5ae93d56281 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:38:09 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:38408
I20260809 03:38:09.838398 26411 tablet_replica.cc:330] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa: stopping tablet replica
I20260809 03:38:09.838938 26411 raft_consensus.cc:2229] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:38:09.839649 26411 raft_consensus.cc:2258] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:38:09.845724 26411 ts_tablet_manager.cc:1905] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:38:09.850159 26411 ts_tablet_manager.cc:1918] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:38:09.850318 26412 tablet_replica.cc:330] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: stopping tablet replica
I20260809 03:38:09.850421 26411 log.cc:1196] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/021406e6fd464b65a907e5ae93d56281
I20260809 03:38:09.850924 26412 raft_consensus.cc:2229] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:38:09.850965 26411 ts_tablet_manager.cc:1939] T 021406e6fd464b65a907e5ae93d56281 P ae95bc86b368469ab62f81dffb6973fa: Deleting consensus metadata
I20260809 03:38:09.851478 26412 raft_consensus.cc:2258] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:38:09.853719 26412 ts_tablet_manager.cc:1905] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:38:09.853979 25653 catalog_manager.cc:4841] TS ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277): tablet 021406e6fd464b65a907e5ae93d56281 (table test-table [id=0b667aab486949f892bc8802ddc99a47]) successfully deleted
I20260809 03:38:09.856745 26412 ts_tablet_manager.cc:1918] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:38:09.856871 26412 log.cc:1196] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/021406e6fd464b65a907e5ae93d56281
I20260809 03:38:09.857302 26412 ts_tablet_manager.cc:1939] T 021406e6fd464b65a907e5ae93d56281 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Deleting consensus metadata
I20260809 03:38:09.858919 25657 catalog_manager.cc:4841] TS 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615): tablet 021406e6fd464b65a907e5ae93d56281 (table test-table [id=0b667aab486949f892bc8802ddc99a47]) successfully deleted
I20260809 03:38:09.860718 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 25952
W20260809 03:38:09.860725 25657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:44633 (error 108)
I20260809 03:38:09.873618 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:09.873252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:44633 (local address 127.0.0.1:44768)
0809 03:38:09.873421 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:09.873522 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:44633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 03:38:09.873958 25657 catalog_manager.cc:4584] TS 8019f869209d4a408923eca695bcf52d (127.1.224.65:44633): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 021406e6fd464b65a907e5ae93d56281: Network error: Client connection negotiation failed: client connection to 127.1.224.65:44633: connect: Connection refused (error 111)
I20260809 03:38:09.880885 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 26135
I20260809 03:38:09.897274 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 25628
I20260809 03:38:29.915223 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37083
--webserver_interface=127.1.224.126
--webserver_port=46707
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:37083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:38:30.040766 26415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:38:30.041148 26415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:38:30.041224 26415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:38:30.045569 26415 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 03:38:30.045675 26415 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 03:38:30.045703 26415 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 03:38:30.045725 26415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:38:30.045773 26415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:38:30.045831 26415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:38:30.045871 26415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:38:30.045905 26415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:38:30.051717 26415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:37083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=46707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:38:30.053155 26415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:38:30.054301 26415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:38:30.069299 26431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:30.069293 26424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:30.069293 26428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:30.069763 26415 server_base.cc:1034] running on GCE node
I20260809 03:38:30.070474 26415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:38:30.071849 26415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:38:30.073055 26415 hybrid_clock.cc:648] HybridClock initialized: now 1786246710073016 us; error 55 us; skew 500 ppm
Aug 09 03:38:30 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:38:30.080264 26415 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:38:30.081396 26415 webserver.cc:460] Webserver started at http://127.1.224.126:46707/ using document root <none> and password file <none>
I20260809 03:38:30.082069 26415 fs_manager.cc:359] Metadata directory not provided
I20260809 03:38:30.082197 26415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:38:30.086622 26415 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 03:38:30.088996 26442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:30.090278 26415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:38:30.090447 26415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "adddaff46a1d4ce7b2e1976fea32f60e"
format_stamp: "Formatted at 2026-08-09 03:38:08 on dist-test-slave-jq4t"
server_key: "04829a2b89634c7b5eaed902167d7d87"
server_key_iv: "b0571bd693da317d284f7099b29db2cf"
server_key_version: "encryptionkey@0"
I20260809 03:38:30.091113 26415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:38:30.114934 26415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:38:30.118539 26415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:38:30.118788 26415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:38:30.127144 26415 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:37083
I20260809 03:38:30.127468 26519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:37083 every 8 connection(s)
I20260809 03:38:30.128449 26415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:38:30.133587 26521 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:38:30.134909 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 26415
I20260809 03:38:30.135406 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:44633
--local_ip_for_outbound_sockets=127.1.224.65
--tserver_master_addrs=127.1.224.126:37083
--webserver_port=40525
--webserver_interface=127.1.224.65
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:38:30.135634 26521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Bootstrap starting.
I20260809 03:38:30.155243 26521 log.cc:824] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Log is configured to *not* fsync() on all Append() calls
I20260809 03:38:30.165191 26521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:38:30.165634 26521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Bootstrap complete.
I20260809 03:38:30.170143 26521 raft_consensus.cc:348] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } }
I20260809 03:38:30.170625 26521 raft_consensus.cc:729] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: adddaff46a1d4ce7b2e1976fea32f60e, State: Initialized, Role: FOLLOWER
I20260809 03:38:30.171319 26521 consensus_queue.cc:260] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } }
I20260809 03:38:30.171479 26521 raft_consensus.cc:388] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:38:30.171584 26521 raft_consensus.cc:482] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:38:30.171711 26521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:38:30.174010 26521 raft_consensus.cc:504] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } }
I20260809 03:38:30.174386 26521 leader_election.cc:302] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adddaff46a1d4ce7b2e1976fea32f60e; no voters:
I20260809 03:38:30.174758 26521 leader_election.cc:288] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 03:38:30.174857 26527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:38:30.175067 26527 raft_consensus.cc:686] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 2 LEADER]: Becoming Leader. State: Replica: adddaff46a1d4ce7b2e1976fea32f60e, State: Running, Role: LEADER
I20260809 03:38:30.175457 26527 consensus_queue.cc:237] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } }
I20260809 03:38:30.176156 26521 sys_catalog.cc:564] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:38:30.177011 26529 sys_catalog.cc:455] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } } }
I20260809 03:38:30.177160 26529 sys_catalog.cc:458] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: This master's current role is: LEADER
I20260809 03:38:30.177580 26530 sys_catalog.cc:455] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: SysCatalogTable state changed. Reason: New leader adddaff46a1d4ce7b2e1976fea32f60e. Latest consensus state: current_term: 2 leader_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } } }
I20260809 03:38:30.177738 26530 sys_catalog.cc:458] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: This master's current role is: LEADER
I20260809 03:38:30.178292 26535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:38:30.183622 26535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:38:30.184439 26535 catalog_manager.cc:1259] Loaded cluster ID: 55025cfb94654a4d919dceab17947e78
I20260809 03:38:30.184602 26535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:38:30.187495 26535 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:38:30.188246 26535 catalog_manager.cc:5879] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Loaded TSK: 0
I20260809 03:38:30.210619 26535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Generated new TSK 1
I20260809 03:38:30.211016 26535 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:38:30.269554 26524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:38:30.269866 26524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:38:30.269915 26524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:38:30.274296 26524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:38:30.274379 26524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:38:30.274442 26524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:38:30.280387 26524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:44633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=40525
--tserver_master_addrs=127.1.224.126:37083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:38:30.281702 26524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:38:30.282807 26524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:38:30.293114 26564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:30.293121 26565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:30.293371 26524 server_base.cc:1034] running on GCE node
W20260809 03:38:30.293658 26567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:30.294253 26524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:38:30.295275 26524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:38:30.296535 26524 hybrid_clock.cc:648] HybridClock initialized: now 1786246710296495 us; error 48 us; skew 500 ppm
Aug 09 03:38:30 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:38:30.300155 26524 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:38:30.301184 26524 webserver.cc:460] Webserver started at http://127.1.224.65:40525/ using document root <none> and password file <none>
I20260809 03:38:30.301795 26524 fs_manager.cc:359] Metadata directory not provided
I20260809 03:38:30.301877 26524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:38:30.306093 26524 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:38:30.308496 26581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:30.309829 26524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:38:30.309991 26524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "8019f869209d4a408923eca695bcf52d"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "f02b770dacf9264186ed67b566f1d205"
server_key_iv: "6cf13acbfea4300899cc60547cce9557"
server_key_version: "encryptionkey@0"
I20260809 03:38:30.310624 26524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:38:30.323930 26524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:38:30.327514 26524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:38:30.327737 26524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:38:30.328397 26524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:38:30.329728 26590 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:38:30.334177 26524 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:38:30.334251 26524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260809 03:38:30.334306 26524 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:38:30.336069 26524 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:38:30.336133 26524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 03:38:30.336413 26590 tablet_bootstrap.cc:492] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Bootstrap starting.
I20260809 03:38:30.348217 26524 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:44633
I20260809 03:38:30.348270 26762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:44633 every 8 connection(s)
I20260809 03:38:30.349684 26524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:38:30.354250 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 26524
I20260809 03:38:30.354807 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:35615
--local_ip_for_outbound_sockets=127.1.224.66
--tserver_master_addrs=127.1.224.126:37083
--webserver_port=44037
--webserver_interface=127.1.224.66
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:38:30.359532 26590 log.cc:824] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:38:30 dist-test-slave-jq4t krb5kdc[25604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:38:30.369210 26590 tablet_bootstrap.cc:492] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:38:30.369755 26590 tablet_bootstrap.cc:492] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Bootstrap complete.
I20260809 03:38:30.371348 26590 ts_tablet_manager.cc:1397] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Time spent bootstrapping tablet: real 0.035s user 0.025s sys 0.008s
I20260809 03:38:30.371820 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.352064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:48477)
0809 03:38:30.352781 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:30.352855 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:30.354090 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:30.355661 (+ 1571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:30.355676 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:30.356230 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:38:30.357067 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.357100 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.358967 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.358980 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.360277 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.360287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.361371 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.361380 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.361654 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.362477 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:38:30.362509 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:38:30.364788 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:38:30.367721 (+ 2933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:30.367731 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:30.367749 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:30.368184 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:30.368712 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:30.368718 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:30.368723 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:30.368940 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:30.369789 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:38:30.369797 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:38:30.370157 (+ 360us) client_negotiation.cc:769] Sending connection context
0809 03:38:30.370349 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 03:38:30.370733 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":157,"threads_started":1}
I20260809 03:38:30.373747 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.352166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:48477 (local address 127.1.224.126:37083)
0809 03:38:30.352757 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:30.352776 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:30.352996 (+ 220us) server_negotiation.cc:407] Connection header received
0809 03:38:30.354662 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:30.354681 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:30.355127 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:30.355528 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:30.357327 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.358759 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.360432 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.361182 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.361366 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.365098 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:30.365105 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:30.365122 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:30.365164 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:30.367553 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:30.368361 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:30.368368 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:30.368373 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:30.368589 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:30.369081 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:30.369084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:30.369086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:30.369649 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:30.369812 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:30.372512 (+ 2700us) server_negotiation.cc:299] Negotiation successful
0809 03:38:30.372858 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":154,"threads_started":1}
I20260809 03:38:30.374032 26590 raft_consensus.cc:348] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } } peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } }
I20260809 03:38:30.374552 26590 raft_consensus.cc:729] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8019f869209d4a408923eca695bcf52d, State: Initialized, Role: FOLLOWER
I20260809 03:38:30.375118 26590 consensus_queue.cc:260] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } } peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } }
I20260809 03:38:30.376394 26763 heartbeater.cc:347] Connected to a master server at 127.1.224.126:37083
I20260809 03:38:30.376720 26763 heartbeater.cc:464] Registering TS with master...
I20260809 03:38:30.377367 26763 heartbeater.cc:511] Master 127.1.224.126:37083 requested a full tablet report, sending...
I20260809 03:38:30.376443 26590 ts_tablet_manager.cc:1428] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:38:30.380273 26466 ts_manager.cc:194] Registered new tserver with Master: 8019f869209d4a408923eca695bcf52d (127.1.224.65:44633)
I20260809 03:38:30.384186 26466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:48477
I20260809 03:38:30.387326 26763 heartbeater.cc:503] Master 127.1.224.126:37083 was elected leader, sending a full tablet report...
Aug 09 03:38:30 dist-test-slave-jq4t krb5kdc[25604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for kudu/127.1.224.65@KRBTEST.COM
I20260809 03:38:30.396963 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.383824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47770 (local address 127.1.224.65:44633)
0809 03:38:30.384340 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:30.384356 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:30.384874 (+ 518us) server_negotiation.cc:407] Connection header received
0809 03:38:30.385113 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:30.385119 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:30.385347 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:30.385541 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:30.386360 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.387570 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.389124 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.389534 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.389677 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.392655 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:30.392667 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:30.392681 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:30.392710 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:30.394473 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:30.395178 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:30.395183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:30.395185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:30.395287 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:30.395652 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:30.395655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:30.395657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:30.396058 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:30.396178 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:30.396640 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 03:38:30.396811 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":132,"threads_started":1}
I20260809 03:38:30.397071 26771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.383701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:44633 (local address 127.0.0.1:47770)
0809 03:38:30.384687 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:30.384737 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:30.384917 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:30.385547 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:30.385554 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:30.386064 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:38:30.386211 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.386220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.387746 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.387750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.388965 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.388977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.389808 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.389811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.389946 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.390620 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:38:30.390633 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:38:30.392432 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:38:30.394598 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:30.394610 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:30.394616 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:30.395011 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:30.395390 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:30.395393 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:30.395395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:30.395503 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:30.396176 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:38:30.396182 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:38:30.396333 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 03:38:30.396506 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 03:38:30.396745 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":128,"threads_started":1}
I20260809 03:38:30.398533 26649 tablet_service.cc:1514] Processing DeleteTablet for tablet 021406e6fd464b65a907e5ae93d56281 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:38:09 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:47770
I20260809 03:38:30.399780 26773 tablet_replica.cc:330] stopping tablet replica
I20260809 03:38:30.400192 26773 raft_consensus.cc:2229] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:38:30.400547 26773 raft_consensus.cc:2258] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:38:30.401686 26773 ts_tablet_manager.cc:1905] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:38:30.405755 26773 ts_tablet_manager.cc:1918] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:38:30.405910 26773 log.cc:1196] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal/wals/021406e6fd464b65a907e5ae93d56281
I20260809 03:38:30.406193 26773 ts_tablet_manager.cc:1939] T 021406e6fd464b65a907e5ae93d56281 P 8019f869209d4a408923eca695bcf52d: Deleting consensus metadata
I20260809 03:38:30.408073 26446 catalog_manager.cc:4841] TS 8019f869209d4a408923eca695bcf52d (127.1.224.65:44633): tablet 021406e6fd464b65a907e5ae93d56281 (table test-table [id=0b667aab486949f892bc8802ddc99a47]) successfully deleted
W20260809 03:38:30.489688 26768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:38:30.489969 26768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:38:30.490015 26768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:38:30.494390 26768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:38:30.494457 26768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:38:30.494519 26768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:38:30.500201 26768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:35615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=44037
--tserver_master_addrs=127.1.224.126:37083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:38:30.501508 26768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:38:30.502628 26768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:38:30.513291 26783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:30.513291 26784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:30.513294 26786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:30.513607 26768 server_base.cc:1034] running on GCE node
I20260809 03:38:30.514314 26768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:38:30.514992 26768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:38:30.516163 26768 hybrid_clock.cc:648] HybridClock initialized: now 1786246710516132 us; error 43 us; skew 500 ppm
Aug 09 03:38:30 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:38:30.519328 26768 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:38:30.520401 26768 webserver.cc:460] Webserver started at http://127.1.224.66:44037/ using document root <none> and password file <none>
I20260809 03:38:30.521116 26768 fs_manager.cc:359] Metadata directory not provided
I20260809 03:38:30.521200 26768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:38:30.525714 26768 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 03:38:30.528143 26797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:30.529721 26768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:38:30.529894 26768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "040c0db80f74b8f474a6c9b59b1fb553"
server_key_iv: "6740a9534066236f963dfd46d48891f4"
server_key_version: "encryptionkey@0"
I20260809 03:38:30.530607 26768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:38:30.545351 26768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:38:30.549021 26768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:38:30.549306 26768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:38:30.550006 26768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:38:30.551038 26768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:38:30.551123 26768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:30.551200 26768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:38:30.551244 26768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:30.561002 26768 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:35615
I20260809 03:38:30.561029 26966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:35615 every 8 connection(s)
I20260809 03:38:30.562115 26768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:38:30.564958 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 26768
I20260809 03:38:30.565510 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:45277
--local_ip_for_outbound_sockets=127.1.224.67
--tserver_master_addrs=127.1.224.126:37083
--webserver_port=33453
--webserver_interface=127.1.224.67
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:38:30 dist-test-slave-jq4t krb5kdc[25604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:38:30.577819 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.564086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:35891 (local address 127.1.224.126:37083)
0809 03:38:30.564238 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:30.564243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:30.565342 (+ 1099us) server_negotiation.cc:407] Connection header received
0809 03:38:30.567293 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:30.567296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:30.567356 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:30.567477 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:30.568785 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.569343 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.570559 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.570812 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.570938 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.573568 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:30.573571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:30.573573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:30.573602 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:30.575200 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:30.575769 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:30.575773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:30.575775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:30.575858 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:30.576306 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:30.576309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:30.576313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:30.576605 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:30.576727 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:30.577554 (+ 827us) server_negotiation.cc:299] Negotiation successful
0809 03:38:30.577685 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:38:30.579056 26969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.564582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:35891)
0809 03:38:30.565143 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:30.565205 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:30.567036 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:30.567660 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:30.567666 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:30.568049 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:38:30.568620 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.568638 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.569485 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.569489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.570407 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.570419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.571008 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.571014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.571185 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.571834 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:38:30.571852 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:38:30.573366 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:38:30.575333 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:30.575341 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:30.575358 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:30.575644 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:30.575966 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:30.575970 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:30.575972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:30.576198 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:30.576754 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:38:30.576764 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:38:30.577241 (+ 477us) client_negotiation.cc:769] Sending connection context
0809 03:38:30.577498 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 03:38:30.577943 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":120,"threads_started":1}
I20260809 03:38:30.580680 26967 heartbeater.cc:347] Connected to a master server at 127.1.224.126:37083
I20260809 03:38:30.580988 26967 heartbeater.cc:464] Registering TS with master...
I20260809 03:38:30.581566 26967 heartbeater.cc:511] Master 127.1.224.126:37083 requested a full tablet report, sending...
I20260809 03:38:30.582759 26466 ts_manager.cc:194] Registered new tserver with Master: 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615)
I20260809 03:38:30.583567 26466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:35891
W20260809 03:38:30.689307 26970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:38:30.689635 26970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:38:30.689713 26970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:38:30.693966 26970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:38:30.694041 26970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:38:30.694104 26970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:38:30.699705 26970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:45277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=33453
--tserver_master_addrs=127.1.224.126:37083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:38:30.700958 26970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:38:30.702107 26970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:38:30.712805 26970 server_base.cc:1034] running on GCE node
W20260809 03:38:30.712785 26978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:30.712710 26979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:30.712714 26982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:30.713480 26970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:38:30.714223 26970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:38:30.715435 26970 hybrid_clock.cc:648] HybridClock initialized: now 1786246710715419 us; error 28 us; skew 500 ppm
Aug 09 03:38:30 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:38:30.718559 26970 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:38:30.719511 26970 webserver.cc:460] Webserver started at http://127.1.224.67:33453/ using document root <none> and password file <none>
I20260809 03:38:30.720139 26970 fs_manager.cc:359] Metadata directory not provided
I20260809 03:38:30.720258 26970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:38:30.724253 26970 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:38:30.726445 26993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:30.727763 26970 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:38:30.727931 26970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "ae95bc86b368469ab62f81dffb6973fa"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "fd2be584eabe5e8ff31bc6616c46473b"
server_key_iv: "acab3cf682fd5ec4e93237c087c89a7d"
server_key_version: "encryptionkey@0"
I20260809 03:38:30.728531 26970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:38:30.746636 26970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:38:30.750319 26970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:38:30.750555 26970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:38:30.751238 26970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:38:30.752156 26970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:38:30.752230 26970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:30.752307 26970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:38:30.752352 26970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:30.763125 26970 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:45277
I20260809 03:38:30.763146 27158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:45277 every 8 connection(s)
I20260809 03:38:30.764354 26970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:38:30.774413 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 26970
Aug 09 03:38:30 dist-test-slave-jq4t krb5kdc[25604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246710, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:38:30.780905 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.766633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:41441 (local address 127.1.224.126:37083)
0809 03:38:30.766798 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:30.766802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:30.767762 (+ 960us) server_negotiation.cc:407] Connection header received
0809 03:38:30.769248 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:30.769252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:30.769323 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:30.769437 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:30.770805 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.771377 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.772812 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.773058 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.773183 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.776599 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:30.776603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:30.776606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:30.776643 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:30.778344 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:30.779020 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:30.779024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:30.779026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:30.779124 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:30.779511 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:30.779515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:30.779517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:30.779722 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:30.779835 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:30.780594 (+ 759us) server_negotiation.cc:299] Negotiation successful
0809 03:38:30.780731 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:38:30.782168 27164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.767052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:41441)
0809 03:38:30.767584 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:30.767633 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:30.768994 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:30.769661 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:30.769668 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:30.770082 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:38:30.770633 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.770647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.771540 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.771543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.772656 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.772668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.773246 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.773250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.773455 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.774238 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:38:30.774266 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:38:30.776366 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:38:30.778475 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:30.778483 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:30.778505 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:30.778899 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:30.779223 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:30.779226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:30.779228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:30.779398 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:30.779837 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:38:30.779847 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:38:30.780219 (+ 372us) client_negotiation.cc:769] Sending connection context
0809 03:38:30.780483 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 03:38:30.780903 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":137,"threads_started":1}
I20260809 03:38:30.783546 27159 heartbeater.cc:347] Connected to a master server at 127.1.224.126:37083
I20260809 03:38:30.783828 27159 heartbeater.cc:464] Registering TS with master...
I20260809 03:38:30.784435 27159 heartbeater.cc:511] Master 127.1.224.126:37083 requested a full tablet report, sending...
I20260809 03:38:30.784802 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.775373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40146 (local address 127.1.224.126:37083)
0809 03:38:30.775661 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:30.775668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:30.775855 (+ 187us) server_negotiation.cc:407] Connection header received
0809 03:38:30.776638 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:30.776642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:30.776691 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:30.776774 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:30.777897 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.778472 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.779409 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.779656 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.779767 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.781289 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:30.781293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:30.781296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:30.781322 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:30.782990 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:30.783458 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:30.783462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:30.783463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:30.783563 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:30.783858 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:30.783862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:30.783864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:30.784082 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:30.784311 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:30.784501 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 03:38:30.784629 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":104,"threads_started":1}
I20260809 03:38:30.785710 26466 ts_manager.cc:194] Registered new tserver with Master: ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277)
I20260809 03:38:30.786495 26466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:41441
I20260809 03:38:30.786845 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:38:30.790916 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.787769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40158 (local address 127.1.224.126:37083)
0809 03:38:30.787887 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:30.787891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:30.787913 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:38:30.788090 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:30.788093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:30.788186 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:30.788269 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:38:30.788567 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.789072 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.789881 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.790103 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.790180 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.790460 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:38:30.790592 (+ 132us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 03:38:30.790802 (+ 210us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40158: token expired
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 03:38:30.791006 27165 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40158: token expired
W20260809 03:38:30.791419 26349 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.224.126:37083)
I20260809 03:38:30.798758 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.791812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40168 (local address 127.1.224.126:37083)
0809 03:38:30.791931 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:30.791934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:30.791958 (+ 24us) server_negotiation.cc:407] Connection header received
0809 03:38:30.792026 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:30.792028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:30.792062 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:30.792125 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:30.792810 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.793295 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.794060 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.794260 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.794334 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.795732 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:30.795735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:30.795737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:30.795758 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:30.797278 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:30.797667 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:30.797670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:30.797671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:30.797744 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:30.798015 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:30.798018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:30.798019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:30.798206 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:30.798321 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:30.798532 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 03:38:30.798646 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:38:30.801364 26466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:38:30.804903 26466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:38:30.814206 26649 tablet_service.cc:1467] Processing CreateTablet for tablet 19904f22eb0945b8a19575b9354ac134 (DEFAULT_TABLE table=test-table [id=df8c6a5525734e7e9181e87f3377e1dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:38:30.815099 26649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19904f22eb0945b8a19575b9354ac134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:38:30.818269 26590 tablet_bootstrap.cc:492] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: Bootstrap starting.
I20260809 03:38:30.818738 26771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.812470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:35615 (local address 127.0.0.1:54398)
0809 03:38:30.812635 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:30.812657 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:30.812748 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:30.813956 (+ 1208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:30.813958 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:30.813979 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:38:30.814104 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.814111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.816036 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.816039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.817031 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.817042 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.818262 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.818266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.818375 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.818436 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 03:38:30.818498 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 03:38:30.818598 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 03:38:30.819115 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.812760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:45277 (local address 127.0.0.1:52500)
0809 03:38:30.813269 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:30.813285 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:30.813387 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:30.814145 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:30.814149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:30.814170 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:38:30.814269 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.814277 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.815960 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.815964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.817217 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.817230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.818723 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.818727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.818814 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.818836 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 03:38:30.818908 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 03:38:30.818981 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":3,"thread_start_us":147,"threads_started":1}
I20260809 03:38:30.819177 27182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.813074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52500 (local address 127.1.224.67:45277)
0809 03:38:30.813652 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:30.813667 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:30.813692 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:38:30.813769 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:30.813774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:30.813977 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:30.814443 (+ 466us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:38:30.814638 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.815821 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.817508 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.818535 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.818695 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.818877 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:30.818939 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 03:38:30.819034 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":103,"threads_started":1}
I20260809 03:38:30.819375 27181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.812896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54398 (local address 127.1.224.66:35615)
0809 03:38:30.813313 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:30.813329 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:30.813357 (+ 28us) server_negotiation.cc:407] Connection header received
0809 03:38:30.813464 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:30.813474 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:30.813728 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:30.813983 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:38:30.814376 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.815897 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.817196 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.818111 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.818911 (+ 800us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.819078 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:30.819132 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 03:38:30.819228 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":174,"threads_started":1}
I20260809 03:38:30.820675 26590 tablet_bootstrap.cc:654] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: Neither blocks nor log segments found. Creating new log.
I20260809 03:38:30.822399 26590 tablet_bootstrap.cc:492] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: No bootstrap required, opened a new log
I20260809 03:38:30.822507 26590 ts_tablet_manager.cc:1397] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260809 03:38:30.822993 26590 raft_consensus.cc:348] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:30.823138 26590 raft_consensus.cc:374] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:38:30.823182 26590 raft_consensus.cc:729] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8019f869209d4a408923eca695bcf52d, State: Initialized, Role: FOLLOWER
I20260809 03:38:30.823400 26590 consensus_queue.cc:260] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:30.823096 26870 tablet_service.cc:1467] Processing CreateTablet for tablet 19904f22eb0945b8a19575b9354ac134 (DEFAULT_TABLE table=test-table [id=df8c6a5525734e7e9181e87f3377e1dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:38:30.823097 27063 tablet_service.cc:1467] Processing CreateTablet for tablet 19904f22eb0945b8a19575b9354ac134 (DEFAULT_TABLE table=test-table [id=df8c6a5525734e7e9181e87f3377e1dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:38:30.823781 26590 ts_tablet_manager.cc:1428] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 03:38:30.824302 26870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19904f22eb0945b8a19575b9354ac134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:38:30.824313 27063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19904f22eb0945b8a19575b9354ac134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:38:30.832173 27184 tablet_bootstrap.cc:492] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Bootstrap starting.
I20260809 03:38:30.832649 27185 tablet_bootstrap.cc:492] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa: Bootstrap starting.
I20260809 03:38:30.835172 27185 tablet_bootstrap.cc:654] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa: Neither blocks nor log segments found. Creating new log.
I20260809 03:38:30.835634 27184 tablet_bootstrap.cc:654] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Neither blocks nor log segments found. Creating new log.
I20260809 03:38:30.836624 27185 log.cc:824] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa: Log is configured to *not* fsync() on all Append() calls
I20260809 03:38:30.836624 27184 log.cc:824] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:38:30.838819 27185 tablet_bootstrap.cc:492] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa: No bootstrap required, opened a new log
I20260809 03:38:30.838819 27184 tablet_bootstrap.cc:492] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: No bootstrap required, opened a new log
I20260809 03:38:30.839053 27184 ts_tablet_manager.cc:1397] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260809 03:38:30.839056 27185 ts_tablet_manager.cc:1397] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:38:30.842079 27185 raft_consensus.cc:348] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:30.842079 27184 raft_consensus.cc:348] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:30.842357 27185 raft_consensus.cc:374] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:38:30.842351 27184 raft_consensus.cc:374] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:38:30.842468 27185 raft_consensus.cc:729] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae95bc86b368469ab62f81dffb6973fa, State: Initialized, Role: FOLLOWER
I20260809 03:38:30.842468 27184 raft_consensus.cc:729] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c3d6d5fdfe248e3b09b68dc8e8fdd5f, State: Initialized, Role: FOLLOWER
I20260809 03:38:30.842942 27185 consensus_queue.cc:260] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:30.842943 27184 consensus_queue.cc:260] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:30.843705 26967 heartbeater.cc:503] Master 127.1.224.126:37083 was elected leader, sending a full tablet report...
I20260809 03:38:30.843883 27159 heartbeater.cc:503] Master 127.1.224.126:37083 was elected leader, sending a full tablet report...
I20260809 03:38:30.844019 27184 ts_tablet_manager.cc:1428] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 03:38:30.844805 27185 ts_tablet_manager.cc:1428] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:38:30.850106 27189 raft_consensus.cc:482] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:38:30.850251 27189 raft_consensus.cc:504] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
W20260809 03:38:30.851106 26764 tablet.cc:2369] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:38:30.851285 27189 leader_election.cc:288] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615), 8019f869209d4a408923eca695bcf52d (127.1.224.65:44633)
I20260809 03:38:30.856261 27181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.852235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:49499 (local address 127.1.224.66:35615)
0809 03:38:30.852367 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:30.852371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:30.852384 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:38:30.852432 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:30.852434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:30.852523 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:30.852598 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:38:30.852860 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.853629 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.855162 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.855671 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.855794 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.855850 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:30.856051 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 03:38:30.856130 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:38:30.856266 27164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.851470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:35615 (local address 127.1.224.67:49499)
0809 03:38:30.851607 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:30.851625 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:30.851712 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:30.852612 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:30.852615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:30.852633 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:38:30.852752 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.852758 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.853754 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.853761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.855031 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.855042 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.855793 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.855797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.855912 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.855936 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:38:30.856020 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 03:38:30.856104 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":27}
I20260809 03:38:30.856300 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.851877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:54911 (local address 127.1.224.65:44633)
0809 03:38:30.852006 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:30.852009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:30.852241 (+ 232us) server_negotiation.cc:407] Connection header received
0809 03:38:30.852388 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:30.852394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:30.852507 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:30.852615 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:38:30.853112 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.853840 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.855002 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.855682 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.855826 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.855992 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:30.856108 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 03:38:30.856187 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:38:30.856321 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.851754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:44633 (local address 127.1.224.67:54911)
0809 03:38:30.852132 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:30.852153 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:30.852270 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:30.852761 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:30.852767 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:30.852791 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:38:30.852960 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.852971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.853950 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.853957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.854838 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:30.854848 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.855841 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.855845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:30.855965 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.855984 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 03:38:30.856092 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 03:38:30.856160 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":26,"thread_start_us":154,"threads_started":1}
I20260809 03:38:30.857071 26681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19904f22eb0945b8a19575b9354ac134" candidate_uuid: "ae95bc86b368469ab62f81dffb6973fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8019f869209d4a408923eca695bcf52d" is_pre_election: true
I20260809 03:38:30.857393 26681 raft_consensus.cc:2454] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae95bc86b368469ab62f81dffb6973fa in term 0.
I20260809 03:38:30.857885 27000 leader_election.cc:302] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8019f869209d4a408923eca695bcf52d, ae95bc86b368469ab62f81dffb6973fa; no voters:
I20260809 03:38:30.857919 26896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19904f22eb0945b8a19575b9354ac134" candidate_uuid: "ae95bc86b368469ab62f81dffb6973fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" is_pre_election: true
I20260809 03:38:30.858183 26896 raft_consensus.cc:2454] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae95bc86b368469ab62f81dffb6973fa in term 0.
I20260809 03:38:30.858193 27189 raft_consensus.cc:2781] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:38:30.858280 27189 raft_consensus.cc:482] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:38:30.858351 27189 raft_consensus.cc:3037] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:38:30.859462 27189 raft_consensus.cc:504] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:30.859812 27189 leader_election.cc:288] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [CANDIDATE]: Term 1 election: Requested vote from peers 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615), 8019f869209d4a408923eca695bcf52d (127.1.224.65:44633)
I20260809 03:38:30.860219 26896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19904f22eb0945b8a19575b9354ac134" candidate_uuid: "ae95bc86b368469ab62f81dffb6973fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f"
I20260809 03:38:30.860309 26681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19904f22eb0945b8a19575b9354ac134" candidate_uuid: "ae95bc86b368469ab62f81dffb6973fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8019f869209d4a408923eca695bcf52d"
I20260809 03:38:30.860350 26896 raft_consensus.cc:3037] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:38:30.860513 26681 raft_consensus.cc:3037] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:38:30.861485 26896 raft_consensus.cc:2454] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae95bc86b368469ab62f81dffb6973fa in term 1.
I20260809 03:38:30.861582 26681 raft_consensus.cc:2454] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae95bc86b368469ab62f81dffb6973fa in term 1.
I20260809 03:38:30.861819 27000 leader_election.cc:302] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c3d6d5fdfe248e3b09b68dc8e8fdd5f, ae95bc86b368469ab62f81dffb6973fa; no voters:
I20260809 03:38:30.862042 27189 raft_consensus.cc:2781] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:38:30.862287 27189 raft_consensus.cc:686] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 1 LEADER]: Becoming Leader. State: Replica: ae95bc86b368469ab62f81dffb6973fa, State: Running, Role: LEADER
I20260809 03:38:30.862639 27189 consensus_queue.cc:237] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:30.866050 26466 catalog_manager.cc:5495] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa reported cstate change: term changed from 0 to 1, leader changed from <none> to ae95bc86b368469ab62f81dffb6973fa (127.1.224.67). New cstate: current_term: 1 leader_uuid: "ae95bc86b368469ab62f81dffb6973fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } health_report { overall_health: UNKNOWN } } }
I20260809 03:38:30.886461 27182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.882948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52510 (local address 127.1.224.67:45277)
0809 03:38:30.883088 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:30.883092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:30.883122 (+ 30us) server_negotiation.cc:407] Connection header received
0809 03:38:30.883290 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:30.883292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:30.883349 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:30.883441 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:38:30.883655 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.884178 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.885046 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:30.885257 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:30.885340 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:30.885599 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:38:30.886036 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:38:30.886129 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:30.886181 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 03:38:30.886233 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:38:30.892815 26681 raft_consensus.cc:1261] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Refusing update from remote peer ae95bc86b368469ab62f81dffb6973fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:38:30.892824 26896 raft_consensus.cc:1261] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 1 FOLLOWER]: Refusing update from remote peer ae95bc86b368469ab62f81dffb6973fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:38:30.893527 27189 consensus_queue.cc:1035] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:38:30.893712 27193 consensus_queue.cc:1035] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:38:30.901859 27199 mvcc.cc:205] Tried to move back new op lower bound from 7316466527809310720 to 7316466527696322560. Current Snapshot: MvccSnapshot[applied={T|T < 7316466527809310720}]
I20260809 03:38:30.912863 26462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:38:30.913079 26462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:38:30.915814 26462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Sending DeleteTablet for 3 replicas of tablet 19904f22eb0945b8a19575b9354ac134
I20260809 03:38:30.916675 27063 tablet_service.cc:1514] Processing DeleteTablet for tablet 19904f22eb0945b8a19575b9354ac134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:38:30 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:52500
I20260809 03:38:30.916724 26870 tablet_service.cc:1514] Processing DeleteTablet for tablet 19904f22eb0945b8a19575b9354ac134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:38:30 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:54398
I20260809 03:38:30.916810 26649 tablet_service.cc:1514] Processing DeleteTablet for tablet 19904f22eb0945b8a19575b9354ac134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:38:30 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:47770
I20260809 03:38:30.917245 27210 tablet_replica.cc:330] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa: stopping tablet replica
I20260809 03:38:30.917480 27209 tablet_replica.cc:330] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: stopping tablet replica
I20260809 03:38:30.917486 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 26524
I20260809 03:38:30.917596 27210 raft_consensus.cc:2229] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:38:30.917663 27209 raft_consensus.cc:2229] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:38:30.917846 27209 raft_consensus.cc:2258] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:38:30.918084 27210 raft_consensus.cc:2258] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:38:30.923588 27210 ts_tablet_manager.cc:1905] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:38:30.926635 27210 ts_tablet_manager.cc:1918] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:38:30.926756 27210 log.cc:1196] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/19904f22eb0945b8a19575b9354ac134
I20260809 03:38:30.927111 27210 ts_tablet_manager.cc:1939] T 19904f22eb0945b8a19575b9354ac134 P ae95bc86b368469ab62f81dffb6973fa: Deleting consensus metadata
I20260809 03:38:30.928495 27212 tablet_replica.cc:330] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: stopping tablet replica
I20260809 03:38:30.928934 27212 raft_consensus.cc:2229] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:38:30.928886 26444 catalog_manager.cc:4841] TS ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277): tablet 19904f22eb0945b8a19575b9354ac134 (table test-table [id=df8c6a5525734e7e9181e87f3377e1dd]) successfully deleted
I20260809 03:38:30.929316 27212 raft_consensus.cc:2258] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:38:30.931457 27212 ts_tablet_manager.cc:1905] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:38:30.934320 27212 ts_tablet_manager.cc:1918] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:38:30.934510 27212 log.cc:1196] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/19904f22eb0945b8a19575b9354ac134
I20260809 03:38:30.934942 27212 ts_tablet_manager.cc:1939] T 19904f22eb0945b8a19575b9354ac134 P 9c3d6d5fdfe248e3b09b68dc8e8fdd5f: Deleting consensus metadata
I20260809 03:38:30.936450 26446 catalog_manager.cc:4841] TS 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615): tablet 19904f22eb0945b8a19575b9354ac134 (table test-table [id=df8c6a5525734e7e9181e87f3377e1dd]) successfully deleted
W20260809 03:38:30.941766 26446 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:44633 (error 108)
I20260809 03:38:30.942549 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 26768
I20260809 03:38:30.943109 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:30.942739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:44633 (local address 127.0.0.1:47772)
0809 03:38:30.942888 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:30.942999 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:44633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 03:38:30.943269 26446 catalog_manager.cc:4584] TS 8019f869209d4a408923eca695bcf52d (127.1.224.65:44633): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19904f22eb0945b8a19575b9354ac134: Network error: Client connection negotiation failed: client connection to 127.1.224.65:44633: connect: Connection refused (error 111)
I20260809 03:38:30.959452 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 26970
I20260809 03:38:30.977522 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 26415
I20260809 03:38:50.998095 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37083
--webserver_interface=127.1.224.126
--webserver_port=46707
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:37083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:38:51.125396 27215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:38:51.125774 27215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:38:51.125885 27215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:38:51.130168 27215 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 03:38:51.130260 27215 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 03:38:51.130287 27215 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 03:38:51.130309 27215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:38:51.130354 27215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:38:51.130399 27215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:38:51.130438 27215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:38:51.130475 27215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:38:51.136250 27215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:37083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=46707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:38:51.137573 27215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:38:51.138744 27215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:38:51.148449 27222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:51.148465 27224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:51.148535 27215 server_base.cc:1034] running on GCE node
W20260809 03:38:51.148454 27226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:51.149454 27215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:38:51.150734 27215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:38:51.151912 27215 hybrid_clock.cc:648] HybridClock initialized: now 1786246731151890 us; error 38 us; skew 500 ppm
Aug 09 03:38:51 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:38:51.158406 27215 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:38:51.159528 27215 webserver.cc:460] Webserver started at http://127.1.224.126:46707/ using document root <none> and password file <none>
I20260809 03:38:51.160193 27215 fs_manager.cc:359] Metadata directory not provided
I20260809 03:38:51.160269 27215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:38:51.164458 27215 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:38:51.166754 27236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:51.168129 27215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 03:38:51.168260 27215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "adddaff46a1d4ce7b2e1976fea32f60e"
format_stamp: "Formatted at 2026-08-09 03:38:08 on dist-test-slave-jq4t"
server_key: "04829a2b89634c7b5eaed902167d7d87"
server_key_iv: "b0571bd693da317d284f7099b29db2cf"
server_key_version: "encryptionkey@0"
I20260809 03:38:51.168885 27215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:38:51.192139 27215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:38:51.195763 27215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:38:51.196009 27215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:38:51.204229 27215 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:37083
I20260809 03:38:51.204599 27317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:37083 every 8 connection(s)
I20260809 03:38:51.205600 27215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:38:51.207125 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 27215
I20260809 03:38:51.207701 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:44633
--local_ip_for_outbound_sockets=127.1.224.65
--tserver_master_addrs=127.1.224.126:37083
--webserver_port=40525
--webserver_interface=127.1.224.65
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:38:51.211694 27318 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:38:51.213780 27318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Bootstrap starting.
I20260809 03:38:51.232923 27318 log.cc:824] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Log is configured to *not* fsync() on all Append() calls
I20260809 03:38:51.244704 27318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:38:51.245178 27318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: Bootstrap complete.
I20260809 03:38:51.249627 27318 raft_consensus.cc:348] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } }
I20260809 03:38:51.250031 27318 raft_consensus.cc:729] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: adddaff46a1d4ce7b2e1976fea32f60e, State: Initialized, Role: FOLLOWER
I20260809 03:38:51.250590 27318 consensus_queue.cc:260] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } }
I20260809 03:38:51.250711 27318 raft_consensus.cc:388] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:38:51.250798 27318 raft_consensus.cc:482] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:38:51.250944 27318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 2 FOLLOWER]: Advancing to term 3
I20260809 03:38:51.253104 27318 raft_consensus.cc:504] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } }
I20260809 03:38:51.253546 27318 leader_election.cc:302] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adddaff46a1d4ce7b2e1976fea32f60e; no voters:
I20260809 03:38:51.253985 27318 leader_election.cc:288] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 03:38:51.254077 27326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 3 FOLLOWER]: Leader election won for term 3
I20260809 03:38:51.254276 27326 raft_consensus.cc:686] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [term 3 LEADER]: Becoming Leader. State: Replica: adddaff46a1d4ce7b2e1976fea32f60e, State: Running, Role: LEADER
I20260809 03:38:51.254649 27326 consensus_queue.cc:237] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } }
I20260809 03:38:51.255299 27318 sys_catalog.cc:564] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:38:51.255661 27326 sys_catalog.cc:455] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: SysCatalogTable state changed. Reason: New leader adddaff46a1d4ce7b2e1976fea32f60e. Latest consensus state: current_term: 3 leader_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } } }
I20260809 03:38:51.255825 27326 sys_catalog.cc:458] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: This master's current role is: LEADER
I20260809 03:38:51.256345 27329 sys_catalog.cc:455] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adddaff46a1d4ce7b2e1976fea32f60e" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37083 } } }
I20260809 03:38:51.256554 27329 sys_catalog.cc:458] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e [sys.catalog]: This master's current role is: LEADER
I20260809 03:38:51.261047 27357 catalog_manager.cc:1259] Loaded cluster ID: 55025cfb94654a4d919dceab17947e78
I20260809 03:38:51.261118 27357 catalog_manager.cc:1552] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: loading cluster ID for follower catalog manager: success
I20260809 03:38:51.263741 27357 catalog_manager.cc:1574] T 00000000000000000000000000000000 P adddaff46a1d4ce7b2e1976fea32f60e: acquiring CA information for follower catalog manager: success
F20260809 03:38:51.265041 27357 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f43cc2cf9ad google::LogMessage::Fail() at ??:0
@ 0x7f43cc2d288a google::LogMessage::SendToLog() at ??:0
@ 0x7f43cc2cf4de google::LogMessage::Flush() at ??:0
@ 0x7f43cc2d0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f43cef9318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f43d6c38989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f43d6c15023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f43d6c1554b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f43d6c0e652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f43d6c0d9e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7f43d6c54541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f43ce512512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f43cfb566db start_thread at ??:0
@ 0x7f43cadcd61f clone at ??:0
W20260809 03:38:51.367182 27321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:38:51.367614 27321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:38:51.367689 27321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:38:51.374296 27321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:38:51.374392 27321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:38:51.374490 27321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:38:51.383173 27321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:44633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=40525
--tserver_master_addrs=127.1.224.126:37083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:38:51.384853 27321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:38:51.386369 27321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:38:51.398773 27389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:51.398833 27393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:51.399212 27321 server_base.cc:1034] running on GCE node
W20260809 03:38:51.399326 27390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:51.399966 27321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:38:51.400921 27321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:38:51.402144 27321 hybrid_clock.cc:648] HybridClock initialized: now 1786246731402109 us; error 49 us; skew 500 ppm
Aug 09 03:38:51 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:38:51.406040 27321 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:38:51.407156 27321 webserver.cc:460] Webserver started at http://127.1.224.65:40525/ using document root <none> and password file <none>
I20260809 03:38:51.407799 27321 fs_manager.cc:359] Metadata directory not provided
I20260809 03:38:51.407863 27321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:38:51.412432 27321 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:38:51.415019 27405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:51.416437 27321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:38:51.416606 27321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "8019f869209d4a408923eca695bcf52d"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "f02b770dacf9264186ed67b566f1d205"
server_key_iv: "6cf13acbfea4300899cc60547cce9557"
server_key_version: "encryptionkey@0"
I20260809 03:38:51.417198 27321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:38:51.433068 27321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:38:51.436725 27321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:38:51.436970 27321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:38:51.437626 27321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:38:51.438823 27418 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:38:51.444579 27321 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:38:51.444653 27321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 03:38:51.444721 27321 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:38:51.447068 27321 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:38:51.447136 27321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 03:38:51.447515 27418 tablet_bootstrap.cc:492] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: Bootstrap starting.
I20260809 03:38:51.459347 27321 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:44633
I20260809 03:38:51.459367 27598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:44633 every 8 connection(s)
I20260809 03:38:51.460667 27321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:38:51.467756 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 27321
I20260809 03:38:51.468336 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:35615
--local_ip_for_outbound_sockets=127.1.224.66
--tserver_master_addrs=127.1.224.126:37083
--webserver_port=44037
--webserver_interface=127.1.224.66
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:38:51.468967 27418 log.cc:824] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:38:51.476570 27418 tablet_bootstrap.cc:492] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:38:51.476974 27418 tablet_bootstrap.cc:492] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: Bootstrap complete.
I20260809 03:38:51.478176 27418 ts_tablet_manager.cc:1397] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: Time spent bootstrapping tablet: real 0.031s user 0.024s sys 0.004s
I20260809 03:38:51.480837 27418 raft_consensus.cc:348] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:51.481316 27418 raft_consensus.cc:729] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8019f869209d4a408923eca695bcf52d, State: Initialized, Role: FOLLOWER
I20260809 03:38:51.481853 27418 consensus_queue.cc:260] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:51.482856 27418 ts_tablet_manager.cc:1428] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260809 03:38:51.542018 27606 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:38:51.463150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:41869)
0809 03:38:51.463838 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:51.463902 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:51.465264 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:51.541139 (+ 75875us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":200,"threads_started":1}
W20260809 03:38:51.542677 27599 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:37083 (0 consecutive failures): Network error: Failed to ping master at 127.1.224.126:37083: Client connection negotiation failed: client connection to 127.1.224.126:37083: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:38:51.543576 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:51.543304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:44791)
0809 03:38:51.543448 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:51.543517 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 03:38:51.544269 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:51.544070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:47555)
0809 03:38:51.544179 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:51.544222 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 03:38:51.544577 27599 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:38:51.609336 27609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:38:51.609630 27609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:38:51.609701 27609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:38:51.614005 27609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:38:51.614068 27609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:38:51.614173 27609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:38:51.621055 27609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:35615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=44037
--tserver_master_addrs=127.1.224.126:37083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:38:51.622308 27609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:38:51.623396 27609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:38:51.633641 27618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:51.633634 27616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:51.633805 27609 server_base.cc:1034] running on GCE node
W20260809 03:38:51.634035 27615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:51.634516 27609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:38:51.635175 27609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:38:51.636375 27609 hybrid_clock.cc:648] HybridClock initialized: now 1786246731636340 us; error 50 us; skew 500 ppm
Aug 09 03:38:51 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:38:51.639333 27609 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:38:51.640323 27609 webserver.cc:460] Webserver started at http://127.1.224.66:44037/ using document root <none> and password file <none>
I20260809 03:38:51.641036 27609 fs_manager.cc:359] Metadata directory not provided
I20260809 03:38:51.641142 27609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:38:51.645358 27609 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:38:51.647639 27635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:51.648932 27609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:38:51.649078 27609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "040c0db80f74b8f474a6c9b59b1fb553"
server_key_iv: "6740a9534066236f963dfd46d48891f4"
server_key_version: "encryptionkey@0"
I20260809 03:38:51.649716 27609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:38:51.668407 27609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:38:51.672181 27609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:38:51.672413 27609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:38:51.673106 27609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:38:51.674062 27609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:38:51.674139 27609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:51.674216 27609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:38:51.674259 27609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:51.684211 27609 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:35615
I20260809 03:38:51.684232 27830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:35615 every 8 connection(s)
I20260809 03:38:51.685357 27609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:38:51.687726 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 27609
I20260809 03:38:51.688261 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:45277
--local_ip_for_outbound_sockets=127.1.224.67
--tserver_master_addrs=127.1.224.126:37083
--webserver_port=33453
--webserver_interface=127.1.224.67
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:42695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:38:51.690026 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:51.687707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:44827)
0809 03:38:51.688328 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:51.689024 (+ 696us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":128,"threads_started":1}
W20260809 03:38:51.690637 27831 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:37083 (0 consecutive failures): Network error: Failed to ping master at 127.1.224.126:37083: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
I20260809 03:38:51.691334 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:51.691093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:57569)
0809 03:38:51.691217 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:51.691269 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 03:38:51.692016 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:51.691791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:58095)
0809 03:38:51.691907 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:51.691959 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 03:38:51.692251 27831 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:38:51.810786 27834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:38:51.811237 27834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:38:51.811328 27834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:38:51.815557 27834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:38:51.815667 27834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:38:51.815771 27834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:38:51.821687 27834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:45277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=33453
--tserver_master_addrs=127.1.224.126:37083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:38:51.823041 27834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:38:51.824249 27834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:38:51.834671 27843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:51.834728 27839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:38:51.834738 27840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:38:51.835165 27834 server_base.cc:1034] running on GCE node
I20260809 03:38:51.835605 27834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:38:51.836313 27834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:38:51.837512 27834 hybrid_clock.cc:648] HybridClock initialized: now 1786246731837474 us; error 53 us; skew 500 ppm
Aug 09 03:38:51 dist-test-slave-jq4t krb5kdc[25604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:38:51.840624 27834 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:38:51.841706 27834 webserver.cc:460] Webserver started at http://127.1.224.67:33453/ using document root <none> and password file <none>
I20260809 03:38:51.842339 27834 fs_manager.cc:359] Metadata directory not provided
I20260809 03:38:51.842413 27834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:38:51.846728 27834 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:38:51.849097 27857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:51.850448 27834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:38:51.850587 27834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "ae95bc86b368469ab62f81dffb6973fa"
format_stamp: "Formatted at 2026-08-09 03:38:09 on dist-test-slave-jq4t"
server_key: "fd2be584eabe5e8ff31bc6616c46473b"
server_key_iv: "acab3cf682fd5ec4e93237c087c89a7d"
server_key_version: "encryptionkey@0"
I20260809 03:38:51.851214 27834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:38:51.869318 27834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:38:51.872602 27834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:38:51.872773 27834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:38:51.873368 27834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:38:51.874228 27834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:38:51.874275 27834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:51.874317 27834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:38:51.874337 27834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:38:51.883816 27834 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:45277
I20260809 03:38:51.883842 28033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:45277 every 8 connection(s)
I20260809 03:38:51.884996 27834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:38:51.886970 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 27834
I20260809 03:38:51.889580 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:51.887351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:50463)
0809 03:38:51.888618 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:51.888755 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":953,"thread_start_us":766,"threads_started":1}
W20260809 03:38:51.890355 28035 heartbeater.cc:644] Failed to heartbeat to 127.1.224.126:37083 (0 consecutive failures): Network error: Failed to ping master at 127.1.224.126:37083: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
I20260809 03:38:51.891170 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:51.890874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:57801)
0809 03:38:51.891043 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:51.891100 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 03:38:51.892057 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:51.891796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:41903)
0809 03:38:51.891947 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:51.891995 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 03:38:51.892421 28035 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 03:38:52.545992 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:52.545424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:45355)
0809 03:38:52.545742 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:52.545834 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":119,"threads_started":1}
I20260809 03:38:52.693300 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:52.692879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:52831)
0809 03:38:52.693137 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:52.693193 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":79,"threads_started":1}
I20260809 03:38:52.893628 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:52.893119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:49529)
0809 03:38:52.893434 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:52.893505 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":96,"threads_started":1}
I20260809 03:38:53.197487 28045 raft_consensus.cc:482] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:38:53.197748 28045 raft_consensus.cc:504] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:53.198987 28045 leader_election.cc:288] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615), ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277)
Aug 09 03:38:53 dist-test-slave-jq4t krb5kdc[25604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.67@KRBTEST.COM
Aug 09 03:38:53 dist-test-slave-jq4t krb5kdc[25604](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246731, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.66@KRBTEST.COM
I20260809 03:38:53.215021 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:53.199309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:45277 (local address 127.1.224.65:41289)
0809 03:38:53.200103 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:53.200115 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:53.200206 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:53.202361 (+ 2155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:53.202364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:53.202560 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:38:53.202800 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:53.202817 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:53.204276 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:53.204280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:53.205497 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:53.205511 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:53.206406 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:53.206410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:53.206619 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:53.207443 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:38:53.207456 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:38:53.209182 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:38:53.211936 (+ 2754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:53.211952 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:53.211967 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:53.212514 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:53.213328 (+ 814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:53.213331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:53.213333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:53.213415 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:53.214059 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:38:53.214063 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:38:53.214291 (+ 228us) client_negotiation.cc:769] Sending connection context
0809 03:38:53.214490 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 03:38:53.214862 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":536,"spinlock_wait_cycles":41984,"thread_start_us":80,"threads_started":1}
I20260809 03:38:53.215185 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:53.199483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:51793 (local address 127.1.224.66:35615)
0809 03:38:53.199839 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:53.199857 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:53.199923 (+ 66us) server_negotiation.cc:407] Connection header received
0809 03:38:53.200929 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:53.200940 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:53.201450 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:53.201819 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:53.203033 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:53.204100 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:53.205809 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:53.206292 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:53.206507 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:53.209750 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:53.209755 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:53.209765 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:53.209798 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:53.211892 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:53.212733 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:53.212739 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:53.212744 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:53.212877 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:53.213296 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:53.213303 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:53.213305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:53.213875 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:53.214054 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:53.214645 (+ 591us) server_negotiation.cc:299] Negotiation successful
0809 03:38:53.214975 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":105,"threads_started":1}
I20260809 03:38:53.215498 28047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:53.199309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:35615 (local address 127.1.224.65:51793)
0809 03:38:53.199711 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:53.199729 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:38:53.199856 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:38:53.201886 (+ 2030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:38:53.201892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:38:53.202294 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:38:53.202800 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:53.202817 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:53.204274 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:53.204278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:53.205659 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:38:53.205672 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:38:53.206458 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:38:53.206462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:38:53.206624 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:53.207391 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:38:53.207409 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:38:53.209558 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:38:53.212109 (+ 2551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:53.212112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:53.212115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:53.212514 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:53.213009 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:38:53.213011 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:38:53.213013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:38:53.213168 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:38:53.214025 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:38:53.214032 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:38:53.214291 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 03:38:53.214492 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 03:38:53.214972 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":389,"spinlock_wait_cycles":7936,"thread_start_us":94,"threads_started":1}
I20260809 03:38:53.215875 28049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:53.199510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:41289 (local address 127.1.224.67:45277)
0809 03:38:53.199952 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0809 03:38:53.199974 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 03:38:53.200217 (+ 243us) server_negotiation.cc:407] Connection header received
0809 03:38:53.201055 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:38:53.201063 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:38:53.201449 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:38:53.201820 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:38:53.203023 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:53.204090 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:53.205663 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:38:53.206260 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:38:53.206458 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:38:53.209509 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:38:53.209518 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:38:53.209540 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:38:53.209578 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:38:53.211790 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:53.213042 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:53.213047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:53.213051 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:53.213244 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:38:53.213540 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:38:53.213542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:38:53.213544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:38:53.213938 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:38:53.214090 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 03:38:53.215401 (+ 1311us) server_negotiation.cc:299] Negotiation successful
0809 03:38:53.215700 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":117,"threads_started":1}
W20260809 03:38:53.218029 27408 leader_election.cc:341] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277): Not found: Tablet not found: 19904f22eb0945b8a19575b9354ac134
W20260809 03:38:53.218219 27414 leader_election.cc:341] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615): Not found: Tablet not found: 19904f22eb0945b8a19575b9354ac134
I20260809 03:38:53.218358 27414 leader_election.cc:302] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8019f869209d4a408923eca695bcf52d; no voters: 9c3d6d5fdfe248e3b09b68dc8e8fdd5f, ae95bc86b368469ab62f81dffb6973fa
I20260809 03:38:53.218673 28045 raft_consensus.cc:2726] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:38:53.547098 28047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:53.546805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:55445)
0809 03:38:53.546949 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:53.547004 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 03:38:53.694594 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:53.694115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:49923)
0809 03:38:53.694407 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:53.694470 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":81,"threads_started":1}
I20260809 03:38:53.895116 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:53.894614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:50429)
0809 03:38:53.894931 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:53.894998 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":86,"threads_started":1}
I20260809 03:38:54.548959 28055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:54.548294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:37507)
0809 03:38:54.548679 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:54.548744 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":129,"threads_started":1}
I20260809 03:38:54.695981 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:54.695475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:35167)
0809 03:38:54.695773 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:54.695873 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":88,"threads_started":1}
I20260809 03:38:54.896589 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:54.895999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:41721)
0809 03:38:54.896334 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:54.896400 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":94,"threads_started":1}
I20260809 03:38:55.016727 28060 raft_consensus.cc:482] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:38:55.016875 28060 raft_consensus.cc:504] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:55.017402 28060 leader_election.cc:288] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615), ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277)
W20260809 03:38:55.018373 27408 leader_election.cc:341] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277): Not found: Tablet not found: 19904f22eb0945b8a19575b9354ac134
W20260809 03:38:55.018532 27414 leader_election.cc:341] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615): Not found: Tablet not found: 19904f22eb0945b8a19575b9354ac134
I20260809 03:38:55.018620 27414 leader_election.cc:302] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8019f869209d4a408923eca695bcf52d; no voters: 9c3d6d5fdfe248e3b09b68dc8e8fdd5f, ae95bc86b368469ab62f81dffb6973fa
I20260809 03:38:55.018808 28060 raft_consensus.cc:2726] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:38:55.550359 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:55.549894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:33149)
0809 03:38:55.550189 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:55.550253 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":81,"threads_started":1}
I20260809 03:38:55.697633 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:55.697139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:58957)
0809 03:38:55.697445 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:55.697515 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":86,"threads_started":1}
I20260809 03:38:55.898178 28065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:55.897605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:36447)
0809 03:38:55.897991 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:55.898067 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":105,"threads_started":1}
I20260809 03:38:56.551954 28066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:56.551442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:43419)
0809 03:38:56.551786 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:56.551850 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":88,"threads_started":1}
I20260809 03:38:56.699113 28069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:56.698553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:51349)
0809 03:38:56.698922 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:56.699004 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":90,"threads_started":1}
I20260809 03:38:56.899736 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:56.899196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:49055)
0809 03:38:56.899549 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:56.899634 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":118,"threads_started":1}
I20260809 03:38:57.553642 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:57.553113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:37813)
0809 03:38:57.553448 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:57.553530 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":102,"threads_started":1}
I20260809 03:38:57.687242 28077 raft_consensus.cc:482] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:38:57.687388 28077 raft_consensus.cc:504] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:38:57.687793 28077 leader_election.cc:288] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615), ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277)
W20260809 03:38:57.688709 27414 leader_election.cc:341] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615): Not found: Tablet not found: 19904f22eb0945b8a19575b9354ac134
W20260809 03:38:57.688808 27408 leader_election.cc:341] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277): Not found: Tablet not found: 19904f22eb0945b8a19575b9354ac134
I20260809 03:38:57.688930 27408 leader_election.cc:302] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8019f869209d4a408923eca695bcf52d; no voters: 9c3d6d5fdfe248e3b09b68dc8e8fdd5f, ae95bc86b368469ab62f81dffb6973fa
I20260809 03:38:57.689113 28077 raft_consensus.cc:2726] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:38:57.700851 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:57.700209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:55225)
0809 03:38:57.700621 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:57.700706 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":90,"threads_started":1}
I20260809 03:38:57.901284 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:57.900773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:57943)
0809 03:38:57.901112 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:57.901181 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":115,"threads_started":1}
I20260809 03:38:58.555049 28082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:58.554500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:38751)
0809 03:38:58.554852 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:58.554938 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":90,"threads_started":1}
I20260809 03:38:58.702395 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:58.701809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:60997)
0809 03:38:58.702222 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:58.702291 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":121,"threads_started":1}
I20260809 03:38:58.902962 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:58.902303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:36039)
0809 03:38:58.902763 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:58.902825 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":106,"threads_started":1}
I20260809 03:38:59.556682 28086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:59.556103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:57929)
0809 03:38:59.556439 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:59.556568 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":135,"threads_started":1}
I20260809 03:38:59.703809 28089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:59.703341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:34355)
0809 03:38:59.703642 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:59.703704 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":97,"threads_started":1}
I20260809 03:38:59.904624 28090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:38:59.904006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:43883)
0809 03:38:59.904349 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 03:38:59.904419 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":113,"threads_started":1}
I20260809 03:39:00.558048 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:00.557563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:40113)
0809 03:39:00.557881 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:00.557941 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":109,"threads_started":1}
I20260809 03:39:00.705181 28093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:00.704726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:44843)
0809 03:39:00.705018 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:00.705076 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":99,"threads_started":1}
I20260809 03:39:00.905967 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:00.905432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:36491)
0809 03:39:00.905803 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:00.905862 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":149,"threads_started":1}
I20260809 03:39:01.521090 28097 raft_consensus.cc:482] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:39:01.521250 28097 raft_consensus.cc:504] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:39:01.521834 28097 leader_election.cc:288] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615), ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277)
W20260809 03:39:01.522867 27414 leader_election.cc:341] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615): Not found: Tablet not found: 19904f22eb0945b8a19575b9354ac134
W20260809 03:39:01.523015 27408 leader_election.cc:341] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277): Not found: Tablet not found: 19904f22eb0945b8a19575b9354ac134
I20260809 03:39:01.523119 27408 leader_election.cc:302] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8019f869209d4a408923eca695bcf52d; no voters: 9c3d6d5fdfe248e3b09b68dc8e8fdd5f, ae95bc86b368469ab62f81dffb6973fa
I20260809 03:39:01.523319 28097 raft_consensus.cc:2726] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:39:01.559659 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:01.559148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:50097)
0809 03:39:01.559483 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:01.559552 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":83,"threads_started":1}
I20260809 03:39:01.706658 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:01.706135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:33529)
0809 03:39:01.706492 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:01.706552 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":112,"threads_started":1}
I20260809 03:39:01.907317 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:01.906869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:59663)
0809 03:39:01.907148 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:01.907208 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":83,"threads_started":1}
I20260809 03:39:02.561208 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:02.560714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:37101)
0809 03:39:02.561027 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:02.561097 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":105,"threads_started":1}
I20260809 03:39:02.708184 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:02.707612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:59403)
0809 03:39:02.707963 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:02.708042 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":85,"threads_started":1}
I20260809 03:39:02.909089 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:02.908386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:49685)
0809 03:39:02.908871 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:02.908943 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":136,"threads_started":1}
I20260809 03:39:03.562569 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:03.562060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:47347)
0809 03:39:03.562384 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:03.562454 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":87,"threads_started":1}
I20260809 03:39:03.709699 28110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:03.709214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:38189)
0809 03:39:03.709521 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:03.709589 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":92,"threads_started":1}
I20260809 03:39:03.910601 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:03.910076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:32833)
0809 03:39:03.910376 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:03.910444 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":86,"threads_started":1}
I20260809 03:39:04.564149 28112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:04.563583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:39881)
0809 03:39:04.563964 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:04.564037 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":94,"threads_started":1}
I20260809 03:39:04.711313 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:04.710755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:52685)
0809 03:39:04.711113 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:04.711186 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":167,"threads_started":1}
I20260809 03:39:04.912333 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:04.911783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:54119)
0809 03:39:04.912113 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:04.912210 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":128,"threads_started":1}
I20260809 03:39:05.095192 28115 raft_consensus.cc:482] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:39:05.095404 28115 raft_consensus.cc:504] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3d6d5fdfe248e3b09b68dc8e8fdd5f" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35615 } } peers { permanent_uuid: "ae95bc86b368469ab62f81dffb6973fa" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45277 } } peers { permanent_uuid: "8019f869209d4a408923eca695bcf52d" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 44633 } }
I20260809 03:39:05.095888 28115 leader_election.cc:288] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615), ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277)
W20260809 03:39:05.096872 27408 leader_election.cc:341] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ae95bc86b368469ab62f81dffb6973fa (127.1.224.67:45277): Not found: Tablet not found: 19904f22eb0945b8a19575b9354ac134
W20260809 03:39:05.096978 27414 leader_election.cc:341] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c3d6d5fdfe248e3b09b68dc8e8fdd5f (127.1.224.66:35615): Not found: Tablet not found: 19904f22eb0945b8a19575b9354ac134
I20260809 03:39:05.097074 27414 leader_election.cc:302] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8019f869209d4a408923eca695bcf52d; no voters: 9c3d6d5fdfe248e3b09b68dc8e8fdd5f, ae95bc86b368469ab62f81dffb6973fa
I20260809 03:39:05.097216 28115 raft_consensus.cc:2726] T 19904f22eb0945b8a19575b9354ac134 P 8019f869209d4a408923eca695bcf52d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:39:05.565723 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:05.565187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:50339)
0809 03:39:05.565538 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:05.565612 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":109,"threads_started":1}
I20260809 03:39:05.712749 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:05.712249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:46569)
0809 03:39:05.712579 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:05.712642 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":91,"threads_started":1}
I20260809 03:39:05.913677 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:05.913208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:44387)
0809 03:39:05.913510 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:05.913572 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":104,"threads_started":1}
I20260809 03:39:06.567560 28121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:06.567021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.65:53523)
0809 03:39:06.567363 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:06.567451 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":94,"threads_started":1}
I20260809 03:39:06.714375 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:06.713831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.66:33837)
0809 03:39:06.714186 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:06.714266 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":103,"threads_started":1}
I20260809 03:39:06.892575 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 27321
I20260809 03:39:06.909531 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 27609
I20260809 03:39:06.918493 28123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:06.917113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:37083 (local address 127.1.224.67:59221)
0809 03:39:06.918240 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:06.918326 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.126:37083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":995,"thread_start_us":815,"threads_started":1}
I20260809 03:39:06.924114 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 27834
2026-08-09T03:39:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59606 ms)
[----------] 30 tests from SecurityITest (252459 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:06 dist-test-slave-jq4t krb5kdc[28130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:06 dist-test-slave-jq4t krb5kdc[28130](info): set up 2 sockets
Aug 09 03:39:06 dist-test-slave-jq4t krb5kdc[28130](info): commencing operation
krb5kdc: starting...
W20260809 03:39:08.211805 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:08 dist-test-slave-jq4t krb5kdc[28130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:39:08.347932 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:40141
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:40141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:08.469784 28152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:08.470135 28152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:08.470223 28152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:08.474359 28152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:08.474475 28152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:08.474511 28152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:08.474555 28152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:08.474586 28152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:08.480036 28152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:40141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:40141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:08.481263 28152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:08.482419 28152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:08.492281 28159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:08.492262 28158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:08.492311 28152 server_base.cc:1034] running on GCE node
W20260809 03:39:08.492259 28161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:08.493343 28152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:08.494674 28152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:08.495886 28152 hybrid_clock.cc:648] HybridClock initialized: now 1786246748495858 us; error 50 us; skew 500 ppm
Aug 09 03:39:08 dist-test-slave-jq4t krb5kdc[28130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246748, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:08.498991 28152 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:39:08.500010 28152 webserver.cc:460] Webserver started at http://127.1.224.126:46429/ using document root <none> and password file <none>
I20260809 03:39:08.500712 28152 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:08.500816 28152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:08.501072 28152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:08.503309 28152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "054446df3938489dacb5f6d7a0e56a3d"
format_stamp: "Formatted at 2026-08-09 03:39:08 on dist-test-slave-jq4t"
server_key: "5de2983d9ec3e6143252e5484395bf8f"
server_key_iv: "a4f1c43144206df09743765a595f0187"
server_key_version: "encryptionkey@0"
I20260809 03:39:08.503917 28152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "054446df3938489dacb5f6d7a0e56a3d"
format_stamp: "Formatted at 2026-08-09 03:39:08 on dist-test-slave-jq4t"
server_key: "5de2983d9ec3e6143252e5484395bf8f"
server_key_iv: "a4f1c43144206df09743765a595f0187"
server_key_version: "encryptionkey@0"
I20260809 03:39:08.508045 28152 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:39:08.510717 28173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:08.512028 28152 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:39:08.512187 28152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "054446df3938489dacb5f6d7a0e56a3d"
format_stamp: "Formatted at 2026-08-09 03:39:08 on dist-test-slave-jq4t"
server_key: "5de2983d9ec3e6143252e5484395bf8f"
server_key_iv: "a4f1c43144206df09743765a595f0187"
server_key_version: "encryptionkey@0"
I20260809 03:39:08.512318 28152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:08.526839 28152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:08.530647 28152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:08.530892 28152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:08.538679 28152 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:40141
I20260809 03:39:08.538774 28252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:40141 every 8 connection(s)
I20260809 03:39:08.539801 28152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:08.543041 28253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:08.546159 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 28152
I20260809 03:39:08.546321 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:08.546619 1921 external_mini_cluster.cc:1428] Setting key 77c8b217b4e9cc3e1878cf6269bf95a5
I20260809 03:39:08.549595 28253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d: Bootstrap starting.
I20260809 03:39:08.552454 28253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:08.553431 28253 log.cc:824] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:08.555392 28253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d: No bootstrap required, opened a new log
Aug 09 03:39:08 dist-test-slave-jq4t krb5kdc[28130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:08.558657 28253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054446df3938489dacb5f6d7a0e56a3d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40141 } }
I20260809 03:39:08.558934 28253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:08.559036 28253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 054446df3938489dacb5f6d7a0e56a3d, State: Initialized, Role: FOLLOWER
I20260809 03:39:08.559556 28253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054446df3938489dacb5f6d7a0e56a3d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40141 } }
I20260809 03:39:08.559719 28253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:08.559805 28253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:08.559928 28253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:08.561342 28253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054446df3938489dacb5f6d7a0e56a3d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40141 } }
I20260809 03:39:08.561786 28253 leader_election.cc:302] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 054446df3938489dacb5f6d7a0e56a3d; no voters:
I20260809 03:39:08.562191 28253 leader_election.cc:288] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:08.562325 28263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:08.562567 28263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [term 1 LEADER]: Becoming Leader. State: Replica: 054446df3938489dacb5f6d7a0e56a3d, State: Running, Role: LEADER
I20260809 03:39:08.562842 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:08.548108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.1.224.126:40141)
0809 03:39:08.548583 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:08.548600 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:08.548638 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:39:08.549265 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:08.549282 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:08.549625 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:08.549974 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:08.550960 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:08.552036 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:08.553289 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:08.553808 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:08.553984 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:08.556575 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:08.556580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:08.556594 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:08.556623 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:08.559051 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:08.559584 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:08.559590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:08.559595 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:08.559745 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:08.560092 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:08.560099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:08.560104 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:08.560724 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:08.560969 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:08.561283 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 03:39:08.561656 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":140,"threads_started":1}
I20260809 03:39:08.562985 28263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054446df3938489dacb5f6d7a0e56a3d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40141 } }
I20260809 03:39:08.563627 28253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:08.566128 28266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 054446df3938489dacb5f6d7a0e56a3d. Latest consensus state: current_term: 1 leader_uuid: "054446df3938489dacb5f6d7a0e56a3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054446df3938489dacb5f6d7a0e56a3d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40141 } } }
I20260809 03:39:08.566588 28266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:08.566325 28265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "054446df3938489dacb5f6d7a0e56a3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054446df3938489dacb5f6d7a0e56a3d" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40141 } } }
I20260809 03:39:08.566958 28265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d [sys.catalog]: This master's current role is: LEADER
W20260809 03:39:08.569015 28281 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:39:08.569101 28281 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:39:08.569201 28282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:08.570108 28282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:08.576990 28282 catalog_manager.cc:1347] Generated new cluster ID: 620ef9d99b2841f0987e63f2acbed9cb
I20260809 03:39:08.577075 28282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:08.595504 28282 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:08.597036 28282 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:08.611330 28282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d: Generated new TSK 0
I20260809 03:39:08.612221 28282 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:39:08.649446 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40141
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:08.800505 28292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:08.800869 28292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:08.800958 28292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:08.805904 28292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:08.806012 28292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:08.806118 28292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:39:08.811865 28292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:08.813158 28292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:08.814353 28292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:39:08.824651 28292 server_base.cc:1034] running on GCE node
W20260809 03:39:08.824630 28300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:08.824709 28303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:08.824635 28299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:08.825251 28292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:08.825872 28292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:08.827059 28292 hybrid_clock.cc:648] HybridClock initialized: now 1786246748827025 us; error 42 us; skew 500 ppm
Aug 09 03:39:08 dist-test-slave-jq4t krb5kdc[28130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246748, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:08.833896 28292 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:39:08.834966 28292 webserver.cc:460] Webserver started at http://127.1.224.65:32797/ using document root <none> and password file <none>
I20260809 03:39:08.835624 28292 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:08.835721 28292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:08.835985 28292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:08.838354 28292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "0c59c136c7c14e6db235fc36b4864be1"
format_stamp: "Formatted at 2026-08-09 03:39:08 on dist-test-slave-jq4t"
server_key: "c589e72b7a6f7507ce2a418a448c5be2"
server_key_iv: "4994869ec777e10d073473b418f8e663"
server_key_version: "encryptionkey@0"
I20260809 03:39:08.838969 28292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "0c59c136c7c14e6db235fc36b4864be1"
format_stamp: "Formatted at 2026-08-09 03:39:08 on dist-test-slave-jq4t"
server_key: "c589e72b7a6f7507ce2a418a448c5be2"
server_key_iv: "4994869ec777e10d073473b418f8e663"
server_key_version: "encryptionkey@0"
I20260809 03:39:08.843263 28292 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 03:39:08.845881 28314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:08.847137 28292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:39:08.847304 28292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "0c59c136c7c14e6db235fc36b4864be1"
format_stamp: "Formatted at 2026-08-09 03:39:08 on dist-test-slave-jq4t"
server_key: "c589e72b7a6f7507ce2a418a448c5be2"
server_key_iv: "4994869ec777e10d073473b418f8e663"
server_key_version: "encryptionkey@0"
I20260809 03:39:08.847430 28292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:08.870971 28292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:08.874238 28292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:08.874478 28292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:08.875186 28292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:08.876132 28292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:08.876205 28292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:08.876281 28292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:08.876322 28292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:08.885892 28292 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:33549
I20260809 03:39:08.885913 28499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:33549 every 8 connection(s)
I20260809 03:39:08.886897 28292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:39:08.889137 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 28292
I20260809 03:39:08.889303 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:39:08.889631 1921 external_mini_cluster.cc:1428] Setting key efa3cd0150455f2de4006ba06ea671c8
Aug 09 03:39:08 dist-test-slave-jq4t krb5kdc[28130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246748, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:39:08.902279 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:08.889000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:45917 (local address 127.1.224.126:40141)
0809 03:39:08.889166 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:08.889172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:08.890233 (+ 1061us) server_negotiation.cc:407] Connection header received
0809 03:39:08.891285 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:08.891289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:08.891343 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:08.891435 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:08.893112 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:08.893859 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:08.895054 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:08.895458 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:08.895585 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:08.898236 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:08.898242 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:08.898247 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:08.898284 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:08.900029 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:08.900581 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:08.900586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:08.900589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:08.900709 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:08.901066 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:08.901070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:08.901072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:08.901333 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:08.901451 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:08.901986 (+ 535us) server_negotiation.cc:299] Negotiation successful
0809 03:39:08.902107 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:39:08.903113 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:08.889503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:40141 (local address 127.1.224.65:45917)
0809 03:39:08.890070 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:08.890127 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:08.891037 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:08.891626 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:08.891636 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:08.892172 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:08.892938 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:08.892960 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:08.894007 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:08.894010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:08.894924 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:08.894934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:08.895607 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:08.895610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:08.895764 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:08.896415 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:08.896432 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:08.898056 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:08.900145 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:08.900150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:08.900163 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:08.900428 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:08.900811 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:08.900814 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:08.900815 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:08.900956 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:08.901464 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:08.901470 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:08.901729 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 03:39:08.901892 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 03:39:08.902169 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":149,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:39:08.904636 28500 heartbeater.cc:347] Connected to a master server at 127.1.224.126:40141
I20260809 03:39:08.904906 28500 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:08.905488 28500 heartbeater.cc:511] Master 127.1.224.126:40141 requested a full tablet report, sending...
I20260809 03:39:08.907124 28197 ts_manager.cc:194] Registered new tserver with Master: 0c59c136c7c14e6db235fc36b4864be1 (127.1.224.65:33549)
I20260809 03:39:08.908306 28197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:45917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:39:08.920218 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40141
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:09.039492 28510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:09.039752 28510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:09.039820 28510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:09.043944 28510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:09.044011 28510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:09.044071 28510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:39:09.049644 28510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:09.050936 28510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:09.052098 28510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:09.062480 28519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:09.062480 28522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:09.062711 28510 server_base.cc:1034] running on GCE node
W20260809 03:39:09.062501 28516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:09.063287 28510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:09.064103 28510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:09.065270 28510 hybrid_clock.cc:648] HybridClock initialized: now 1786246749065197 us; error 85 us; skew 500 ppm
Aug 09 03:39:09 dist-test-slave-jq4t krb5kdc[28130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246749, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:09.068295 28510 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:39:09.069370 28510 webserver.cc:460] Webserver started at http://127.1.224.66:37881/ using document root <none> and password file <none>
I20260809 03:39:09.070019 28510 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:09.070118 28510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:09.070374 28510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:09.072695 28510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "645e10afc2b547e79e667606cf6f68d4"
format_stamp: "Formatted at 2026-08-09 03:39:09 on dist-test-slave-jq4t"
server_key: "1378587d15b53fa84fa45b8b4dc5fac5"
server_key_iv: "1e64c17e78852283df470aa19283bf38"
server_key_version: "encryptionkey@0"
I20260809 03:39:09.073314 28510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "645e10afc2b547e79e667606cf6f68d4"
format_stamp: "Formatted at 2026-08-09 03:39:09 on dist-test-slave-jq4t"
server_key: "1378587d15b53fa84fa45b8b4dc5fac5"
server_key_iv: "1e64c17e78852283df470aa19283bf38"
server_key_version: "encryptionkey@0"
I20260809 03:39:09.077564 28510 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:39:09.080374 28529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:09.081874 28510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:09.082060 28510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "645e10afc2b547e79e667606cf6f68d4"
format_stamp: "Formatted at 2026-08-09 03:39:09 on dist-test-slave-jq4t"
server_key: "1378587d15b53fa84fa45b8b4dc5fac5"
server_key_iv: "1e64c17e78852283df470aa19283bf38"
server_key_version: "encryptionkey@0"
I20260809 03:39:09.082201 28510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:09.102276 28510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:09.105551 28510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:09.105784 28510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:09.106441 28510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:09.107344 28510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:09.107414 28510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:09.107491 28510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:09.107543 28510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:09.117502 28510 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:42243
I20260809 03:39:09.117528 28702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:42243 every 8 connection(s)
I20260809 03:39:09.118559 28510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
Aug 09 03:39:09 dist-test-slave-jq4t krb5kdc[28130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246749, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:09.128271 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 28510
I20260809 03:39:09.128417 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:39:09.128782 1921 external_mini_cluster.cc:1428] Setting key 395272573f9f1582658e71a167efd0ef
I20260809 03:39:09.132678 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.120459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:60383 (local address 127.1.224.126:40141)
0809 03:39:09.120656 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:09.120661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:09.121616 (+ 955us) server_negotiation.cc:407] Connection header received
0809 03:39:09.122501 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:09.122504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:09.122557 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:09.122634 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:09.123844 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.124389 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.125546 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.125772 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.125852 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.128642 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:09.128645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:09.128648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:09.128670 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:09.130445 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:09.131001 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:09.131004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:09.131006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:09.131099 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:09.131464 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:09.131467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:09.131469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:09.131662 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:09.131767 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:09.132322 (+ 555us) server_negotiation.cc:299] Negotiation successful
0809 03:39:09.132467 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 03:39:09.133539 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.120949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:40141 (local address 127.1.224.66:60383)
0809 03:39:09.121437 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:09.121485 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:09.122295 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:09.122810 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:09.122818 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:09.123195 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:09.123712 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.123726 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.124520 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.124522 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.125435 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.125445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.125911 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.125914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.126071 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.126737 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:09.126754 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:09.128429 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:09.130573 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:09.130580 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:09.130600 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:09.130882 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:09.131191 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:09.131195 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:09.131197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:09.131347 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:09.131771 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:09.131780 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:09.132063 (+ 283us) client_negotiation.cc:769] Sending connection context
0809 03:39:09.132259 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 03:39:09.132642 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":121,"threads_started":1}
I20260809 03:39:09.134842 28703 heartbeater.cc:347] Connected to a master server at 127.1.224.126:40141
I20260809 03:39:09.135113 28703 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:09.135632 28703 heartbeater.cc:511] Master 127.1.224.126:40141 requested a full tablet report, sending...
I20260809 03:39:09.136834 28197 ts_manager.cc:194] Registered new tserver with Master: 645e10afc2b547e79e667606cf6f68d4 (127.1.224.66:42243)
I20260809 03:39:09.137442 28197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:60383
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:39:09.157299 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40141
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:33909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:09.277742 28711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:09.278064 28711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:09.278108 28711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:09.282235 28711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:09.282305 28711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:09.282363 28711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:39:09.287900 28711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:33909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:40141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:09.289111 28711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:09.290138 28711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:09.300401 28723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:09.300572 28711 server_base.cc:1034] running on GCE node
W20260809 03:39:09.300413 28717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:09.300413 28719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:09.301227 28711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:09.301887 28711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:09.303054 28711 hybrid_clock.cc:648] HybridClock initialized: now 1786246749303038 us; error 29 us; skew 500 ppm
Aug 09 03:39:09 dist-test-slave-jq4t krb5kdc[28130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246749, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:09.306031 28711 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:39:09.306964 28711 webserver.cc:460] Webserver started at http://127.1.224.67:37967/ using document root <none> and password file <none>
I20260809 03:39:09.307585 28711 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:09.307680 28711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:09.307925 28711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:09.310166 28711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "73f803b8214e480ead97b5b4f2c3ba89"
format_stamp: "Formatted at 2026-08-09 03:39:09 on dist-test-slave-jq4t"
server_key: "6260d3880c3375c6a30029edba3b3284"
server_key_iv: "e2fcf8fa896ac92b4661b09588dfca8b"
server_key_version: "encryptionkey@0"
I20260809 03:39:09.310849 28711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "73f803b8214e480ead97b5b4f2c3ba89"
format_stamp: "Formatted at 2026-08-09 03:39:09 on dist-test-slave-jq4t"
server_key: "6260d3880c3375c6a30029edba3b3284"
server_key_iv: "e2fcf8fa896ac92b4661b09588dfca8b"
server_key_version: "encryptionkey@0"
I20260809 03:39:09.314744 28711 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:39:09.317549 28731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:09.318811 28711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:39:09.318966 28711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "73f803b8214e480ead97b5b4f2c3ba89"
format_stamp: "Formatted at 2026-08-09 03:39:09 on dist-test-slave-jq4t"
server_key: "6260d3880c3375c6a30029edba3b3284"
server_key_iv: "e2fcf8fa896ac92b4661b09588dfca8b"
server_key_version: "encryptionkey@0"
I20260809 03:39:09.319085 28711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:09.338490 28711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:09.341897 28711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:09.342135 28711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:09.342810 28711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:09.343904 28711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:09.343983 28711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:09.344059 28711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:09.344098 28711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:09.353382 28711 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:43861
I20260809 03:39:09.353407 28914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:43861 every 8 connection(s)
I20260809 03:39:09.354354 28711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:39:09.355064 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 28711
I20260809 03:39:09.355221 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:39:09.355604 1921 external_mini_cluster.cc:1428] Setting key 484af9a226195fec892a03c7901118ae
Aug 09 03:39:09 dist-test-slave-jq4t krb5kdc[28130](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246749, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:09.369180 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.356416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:59001 (local address 127.1.224.126:40141)
0809 03:39:09.356584 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:09.356589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:09.358226 (+ 1637us) server_negotiation.cc:407] Connection header received
0809 03:39:09.359130 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:09.359133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:09.359192 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:09.359270 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:09.360501 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.361069 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.362321 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.362564 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.362670 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.365468 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:09.365473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:09.365475 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:09.365509 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:09.366793 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:09.367341 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:09.367344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:09.367346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:09.367434 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:09.367806 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:09.367810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:09.367811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:09.368032 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:09.368148 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:09.368884 (+ 736us) server_negotiation.cc:299] Negotiation successful
0809 03:39:09.369017 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 03:39:09.370522 28920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.356975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:40141 (local address 127.1.224.67:59001)
0809 03:39:09.358066 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:09.358139 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:09.358923 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:09.359448 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:09.359457 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:09.359813 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:09.360315 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.360328 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.361202 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.361206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.362185 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.362197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.362728 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.362733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.362913 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.363563 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:09.363581 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:09.365277 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:09.366918 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:09.366924 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:09.366942 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:09.367227 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:09.367544 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:09.367548 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:09.367549 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:09.367704 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:09.368148 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:09.368157 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:09.368579 (+ 422us) client_negotiation.cc:769] Sending connection context
0809 03:39:09.368824 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 03:39:09.369499 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":119,"threads_started":1}
I20260809 03:39:09.371692 28916 heartbeater.cc:347] Connected to a master server at 127.1.224.126:40141
I20260809 03:39:09.371929 28916 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:09.372428 28916 heartbeater.cc:511] Master 127.1.224.126:40141 requested a full tablet report, sending...
I20260809 03:39:09.373579 28197 ts_manager.cc:194] Registered new tserver with Master: 73f803b8214e480ead97b5b4f2c3ba89 (127.1.224.67:43861)
I20260809 03:39:09.374159 28197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:59001
I20260809 03:39:09.382754 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:39:09.394441 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.385441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42470 (local address 127.1.224.126:40141)
0809 03:39:09.385619 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:09.385624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:09.386473 (+ 849us) server_negotiation.cc:407] Connection header received
0809 03:39:09.386721 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:09.386724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:09.386780 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:09.386872 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:09.387704 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.388283 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.389217 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.389455 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.389566 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.391073 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:09.391077 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:09.391080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:09.391116 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:09.392663 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:09.393100 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:09.393104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:09.393106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:09.393195 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:09.393480 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:09.393484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:09.393486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:09.393744 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:09.393892 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:09.394143 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 03:39:09.394264 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:39:09.397689 28197 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:39:09.400996 28197 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:39:09.416461 28935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.410351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33549 (local address 127.0.0.1:59132)
0809 03:39:09.411397 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:09.411412 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:09.411516 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:09.412319 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:09.412322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:09.412336 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:09.412441 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.412449 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.413535 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.413541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.414839 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.414849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.415867 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.415871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.416000 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.416041 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 03:39:09.416162 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 03:39:09.416263 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"mutex_wait_us":94,"thread_start_us":146,"threads_started":1}
I20260809 03:39:09.416976 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.410351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:42243 (local address 127.0.0.1:47908)
0809 03:39:09.411243 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:09.411303 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:09.411458 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:09.412177 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:09.412184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:09.412202 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:09.412350 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.412357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.413960 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.413962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.415301 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.415312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.416401 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.416405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.416526 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.416668 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 03:39:09.416751 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 03:39:09.416831 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":194,"thread_start_us":121,"threads_started":1}
I20260809 03:39:09.417080 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.410488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47908 (local address 127.1.224.66:42243)
0809 03:39:09.411250 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:09.411264 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:09.411606 (+ 342us) server_negotiation.cc:407] Connection header received
0809 03:39:09.411705 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:09.411712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:09.411945 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:09.412106 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:09.412465 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.413746 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.415435 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.416266 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.416435 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.416625 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:09.416798 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 03:39:09.416920 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":119,"threads_started":1}
I20260809 03:39:09.417325 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.411615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49864 (local address 127.1.224.67:43861)
0809 03:39:09.412064 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:09.412074 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:09.412124 (+ 50us) server_negotiation.cc:407] Connection header received
0809 03:39:09.412203 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:09.412211 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:09.412379 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:09.412570 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:09.412722 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.413808 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.415410 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.416266 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.416438 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.416612 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:09.417073 (+ 461us) server_negotiation.cc:299] Negotiation successful
0809 03:39:09.417182 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":148,"threads_started":1}
I20260809 03:39:09.417443 28936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.410645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59132 (local address 127.1.224.65:33549)
0809 03:39:09.411786 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:09.411794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:09.411819 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:39:09.411895 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:09.411902 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:09.412115 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:09.412261 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:09.412608 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.413412 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.414971 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.415731 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.415861 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.416030 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:09.417028 (+ 998us) server_negotiation.cc:299] Negotiation successful
0809 03:39:09.417277 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1016,"thread_start_us":128,"threads_started":1}
I20260809 03:39:09.417598 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.411132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:43861 (local address 127.0.0.1:49864)
0809 03:39:09.411962 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:09.411975 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:09.412063 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:09.412534 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:09.412540 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:09.412561 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:09.412639 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.412644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.413948 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.413952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.415248 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.415261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.416693 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.416697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.416813 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.416837 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:39:09.416910 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 03:39:09.416981 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":5,"thread_start_us":100,"threads_started":1}
I20260809 03:39:09.420325 28808 tablet_service.cc:1467] Processing CreateTablet for tablet 7266eb74523740e5a4c79c830b0aa0e5 (DEFAULT_TABLE table=test-table [id=9a8eaf8ebb0740aaad1dda25213b97b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:09.420428 28396 tablet_service.cc:1467] Processing CreateTablet for tablet 7266eb74523740e5a4c79c830b0aa0e5 (DEFAULT_TABLE table=test-table [id=9a8eaf8ebb0740aaad1dda25213b97b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:09.420408 28598 tablet_service.cc:1467] Processing CreateTablet for tablet 7266eb74523740e5a4c79c830b0aa0e5 (DEFAULT_TABLE table=test-table [id=9a8eaf8ebb0740aaad1dda25213b97b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:09.421552 28396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7266eb74523740e5a4c79c830b0aa0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:09.421552 28598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7266eb74523740e5a4c79c830b0aa0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:09.421581 28808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7266eb74523740e5a4c79c830b0aa0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:09.428375 28942 tablet_bootstrap.cc:492] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89: Bootstrap starting.
I20260809 03:39:09.429854 28946 tablet_bootstrap.cc:492] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4: Bootstrap starting.
I20260809 03:39:09.429854 28944 tablet_bootstrap.cc:492] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1: Bootstrap starting.
I20260809 03:39:09.430721 28942 tablet_bootstrap.cc:654] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:09.432116 28944 tablet_bootstrap.cc:654] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:09.432075 28942 log.cc:824] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:09.432559 28946 tablet_bootstrap.cc:654] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:09.433053 28944 log.cc:824] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:09.433322 28946 log.cc:824] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:09.434649 28942 tablet_bootstrap.cc:492] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89: No bootstrap required, opened a new log
I20260809 03:39:09.434860 28942 ts_tablet_manager.cc:1397] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 03:39:09.434985 28944 tablet_bootstrap.cc:492] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1: No bootstrap required, opened a new log
I20260809 03:39:09.435194 28944 ts_tablet_manager.cc:1397] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:39:09.435292 28946 tablet_bootstrap.cc:492] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4: No bootstrap required, opened a new log
I20260809 03:39:09.435470 28946 ts_tablet_manager.cc:1397] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:39:09.438761 28944 raft_consensus.cc:348] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645e10afc2b547e79e667606cf6f68d4" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42243 } } peers { permanent_uuid: "0c59c136c7c14e6db235fc36b4864be1" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33549 } } peers { permanent_uuid: "73f803b8214e480ead97b5b4f2c3ba89" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43861 } }
I20260809 03:39:09.439038 28944 raft_consensus.cc:374] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:09.438879 28942 raft_consensus.cc:348] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645e10afc2b547e79e667606cf6f68d4" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42243 } } peers { permanent_uuid: "0c59c136c7c14e6db235fc36b4864be1" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33549 } } peers { permanent_uuid: "73f803b8214e480ead97b5b4f2c3ba89" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43861 } }
I20260809 03:39:09.439095 28944 raft_consensus.cc:729] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c59c136c7c14e6db235fc36b4864be1, State: Initialized, Role: FOLLOWER
I20260809 03:39:09.439522 28942 raft_consensus.cc:374] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:09.439595 28942 raft_consensus.cc:729] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f803b8214e480ead97b5b4f2c3ba89, State: Initialized, Role: FOLLOWER
I20260809 03:39:09.439610 28944 consensus_queue.cc:260] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645e10afc2b547e79e667606cf6f68d4" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42243 } } peers { permanent_uuid: "0c59c136c7c14e6db235fc36b4864be1" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33549 } } peers { permanent_uuid: "73f803b8214e480ead97b5b4f2c3ba89" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43861 } }
I20260809 03:39:09.439759 28946 raft_consensus.cc:348] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645e10afc2b547e79e667606cf6f68d4" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42243 } } peers { permanent_uuid: "0c59c136c7c14e6db235fc36b4864be1" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33549 } } peers { permanent_uuid: "73f803b8214e480ead97b5b4f2c3ba89" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43861 } }
I20260809 03:39:09.440032 28946 raft_consensus.cc:374] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:09.440096 28946 raft_consensus.cc:729] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 645e10afc2b547e79e667606cf6f68d4, State: Initialized, Role: FOLLOWER
I20260809 03:39:09.440050 28942 consensus_queue.cc:260] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645e10afc2b547e79e667606cf6f68d4" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42243 } } peers { permanent_uuid: "0c59c136c7c14e6db235fc36b4864be1" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33549 } } peers { permanent_uuid: "73f803b8214e480ead97b5b4f2c3ba89" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43861 } }
I20260809 03:39:09.440611 28500 heartbeater.cc:503] Master 127.1.224.126:40141 was elected leader, sending a full tablet report...
I20260809 03:39:09.440559 28946 consensus_queue.cc:260] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645e10afc2b547e79e667606cf6f68d4" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42243 } } peers { permanent_uuid: "0c59c136c7c14e6db235fc36b4864be1" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33549 } } peers { permanent_uuid: "73f803b8214e480ead97b5b4f2c3ba89" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43861 } }
I20260809 03:39:09.440776 28944 ts_tablet_manager.cc:1428] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 03:39:09.441069 28942 ts_tablet_manager.cc:1428] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260809 03:39:09.441416 28916 heartbeater.cc:503] Master 127.1.224.126:40141 was elected leader, sending a full tablet report...
I20260809 03:39:09.441426 28703 heartbeater.cc:503] Master 127.1.224.126:40141 was elected leader, sending a full tablet report...
I20260809 03:39:09.441485 28946 ts_tablet_manager.cc:1428] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260809 03:39:09.605561 28919 tablet.cc:2369] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:39:09.619866 28704 tablet.cc:2369] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:39:09.638437 28502 tablet.cc:2369] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:39:09.754731 28955 raft_consensus.cc:482] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:39:09.755002 28955 raft_consensus.cc:504] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645e10afc2b547e79e667606cf6f68d4" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42243 } } peers { permanent_uuid: "0c59c136c7c14e6db235fc36b4864be1" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33549 } } peers { permanent_uuid: "73f803b8214e480ead97b5b4f2c3ba89" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43861 } }
I20260809 03:39:09.756438 28955 leader_election.cc:288] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c59c136c7c14e6db235fc36b4864be1 (127.1.224.65:33549), 73f803b8214e480ead97b5b4f2c3ba89 (127.1.224.67:43861)
I20260809 03:39:09.760828 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.756957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:36669 (local address 127.1.224.67:43861)
0809 03:39:09.757121 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:09.757126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:09.757355 (+ 229us) server_negotiation.cc:407] Connection header received
0809 03:39:09.757509 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:09.757512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:09.757554 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:09.757658 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:09.757999 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.758490 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.759755 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.760172 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.760303 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.760364 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:09.760595 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 03:39:09.760674 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 03:39:09.760854 28936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.756905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:48005 (local address 127.1.224.65:33549)
0809 03:39:09.757076 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:09.757081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:09.757304 (+ 223us) server_negotiation.cc:407] Connection header received
0809 03:39:09.757482 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:09.757486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:09.757544 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:09.757610 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:09.757946 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.758493 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.759733 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.760168 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.760314 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.760374 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:09.760631 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 03:39:09.760720 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 03:39:09.760872 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.756836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:43861 (local address 127.1.224.66:36669)
0809 03:39:09.757219 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:09.757231 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:09.757374 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:09.757699 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:09.757703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:09.757720 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:09.757856 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.757861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.758614 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.758619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.759595 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.759605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.760315 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.760319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.760468 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.760505 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 03:39:09.760589 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 03:39:09.760692 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":70,"thread_start_us":137,"threads_started":1}
I20260809 03:39:09.760857 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.756740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33549 (local address 127.1.224.66:48005)
0809 03:39:09.757180 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:09.757194 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:09.757347 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:09.757655 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:09.757658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:09.757674 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:09.757818 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.757825 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.758650 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.758653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.759591 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:09.759602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.760303 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.760307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:09.760423 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.760519 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 03:39:09.760596 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 03:39:09.760691 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":66,"thread_start_us":107,"threads_started":1}
I20260809 03:39:09.761759 28851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7266eb74523740e5a4c79c830b0aa0e5" candidate_uuid: "645e10afc2b547e79e667606cf6f68d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f803b8214e480ead97b5b4f2c3ba89" is_pre_election: true
I20260809 03:39:09.761909 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7266eb74523740e5a4c79c830b0aa0e5" candidate_uuid: "645e10afc2b547e79e667606cf6f68d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c59c136c7c14e6db235fc36b4864be1" is_pre_election: true
I20260809 03:39:09.762172 28430 raft_consensus.cc:2454] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 645e10afc2b547e79e667606cf6f68d4 in term 0.
I20260809 03:39:09.762172 28851 raft_consensus.cc:2454] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 645e10afc2b547e79e667606cf6f68d4 in term 0.
I20260809 03:39:09.762708 28535 leader_election.cc:302] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 645e10afc2b547e79e667606cf6f68d4, 73f803b8214e480ead97b5b4f2c3ba89; no voters:
I20260809 03:39:09.762986 28955 raft_consensus.cc:2781] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:39:09.763105 28955 raft_consensus.cc:482] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:39:09.763175 28955 raft_consensus.cc:3037] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:09.764460 28955 raft_consensus.cc:504] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645e10afc2b547e79e667606cf6f68d4" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42243 } } peers { permanent_uuid: "0c59c136c7c14e6db235fc36b4864be1" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33549 } } peers { permanent_uuid: "73f803b8214e480ead97b5b4f2c3ba89" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43861 } }
I20260809 03:39:09.764947 28955 leader_election.cc:288] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [CANDIDATE]: Term 1 election: Requested vote from peers 0c59c136c7c14e6db235fc36b4864be1 (127.1.224.65:33549), 73f803b8214e480ead97b5b4f2c3ba89 (127.1.224.67:43861)
I20260809 03:39:09.765395 28851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7266eb74523740e5a4c79c830b0aa0e5" candidate_uuid: "645e10afc2b547e79e667606cf6f68d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f803b8214e480ead97b5b4f2c3ba89"
I20260809 03:39:09.765471 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7266eb74523740e5a4c79c830b0aa0e5" candidate_uuid: "645e10afc2b547e79e667606cf6f68d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c59c136c7c14e6db235fc36b4864be1"
I20260809 03:39:09.765573 28851 raft_consensus.cc:3037] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:09.765600 28430 raft_consensus.cc:3037] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:09.766999 28430 raft_consensus.cc:2454] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 645e10afc2b547e79e667606cf6f68d4 in term 1.
I20260809 03:39:09.766999 28851 raft_consensus.cc:2454] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 645e10afc2b547e79e667606cf6f68d4 in term 1.
I20260809 03:39:09.767340 28535 leader_election.cc:302] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 645e10afc2b547e79e667606cf6f68d4, 73f803b8214e480ead97b5b4f2c3ba89; no voters:
I20260809 03:39:09.767525 28955 raft_consensus.cc:2781] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:09.767776 28955 raft_consensus.cc:686] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 1 LEADER]: Becoming Leader. State: Replica: 645e10afc2b547e79e667606cf6f68d4, State: Running, Role: LEADER
I20260809 03:39:09.768203 28955 consensus_queue.cc:237] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645e10afc2b547e79e667606cf6f68d4" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42243 } } peers { permanent_uuid: "0c59c136c7c14e6db235fc36b4864be1" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33549 } } peers { permanent_uuid: "73f803b8214e480ead97b5b4f2c3ba89" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43861 } }
I20260809 03:39:09.771720 28197 catalog_manager.cc:5495] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 645e10afc2b547e79e667606cf6f68d4 (127.1.224.66). New cstate: current_term: 1 leader_uuid: "645e10afc2b547e79e667606cf6f68d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645e10afc2b547e79e667606cf6f68d4" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 42243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c59c136c7c14e6db235fc36b4864be1" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73f803b8214e480ead97b5b4f2c3ba89" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43861 } health_report { overall_health: UNKNOWN } } }
I20260809 03:39:09.870285 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.866728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47912 (local address 127.1.224.66:42243)
0809 03:39:09.866889 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:09.866893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:09.866907 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:39:09.866994 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:09.866997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:09.867060 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:09.867155 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:09.867382 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.867904 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.868904 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:09.869134 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:09.869218 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:09.869488 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:09.869954 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:09.870051 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:09.870107 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 03:39:09.870162 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:39:09.877213 28430 raft_consensus.cc:1261] T 7266eb74523740e5a4c79c830b0aa0e5 P 0c59c136c7c14e6db235fc36b4864be1 [term 1 FOLLOWER]: Refusing update from remote peer 645e10afc2b547e79e667606cf6f68d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:09.877213 28851 raft_consensus.cc:1261] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89 [term 1 FOLLOWER]: Refusing update from remote peer 645e10afc2b547e79e667606cf6f68d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:09.878212 28955 consensus_queue.cc:1035] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c59c136c7c14e6db235fc36b4864be1" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 33549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:09.878511 28962 consensus_queue.cc:1035] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73f803b8214e480ead97b5b4f2c3ba89" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 43861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:09.890038 28968 mvcc.cc:205] Tried to move back new op lower bound from 7316466687487098880 to 7316466687053971456. Current Snapshot: MvccSnapshot[applied={T|T < 7316466687487098880}]
I20260809 03:39:09.892002 28967 mvcc.cc:205] Tried to move back new op lower bound from 7316466687487098880 to 7316466687053971456. Current Snapshot: MvccSnapshot[applied={T|T < 7316466687487098880 or (T in {7316466687487098880})}]
I20260809 03:39:09.897513 28197 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:39:09.897716 28197 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:39:09.900207 28197 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 054446df3938489dacb5f6d7a0e56a3d: Sending DeleteTablet for 3 replicas of tablet 7266eb74523740e5a4c79c830b0aa0e5
I20260809 03:39:09.901096 28598 tablet_service.cc:1514] Processing DeleteTablet for tablet 7266eb74523740e5a4c79c830b0aa0e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:09 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:47908
I20260809 03:39:09.901628 28977 tablet_replica.cc:330] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4: stopping tablet replica
I20260809 03:39:09.902079 28977 raft_consensus.cc:2229] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:39:09.902083 28808 tablet_service.cc:1514] Processing DeleteTablet for tablet 7266eb74523740e5a4c79c830b0aa0e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:09 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:49864
I20260809 03:39:09.902200 28396 tablet_service.cc:1514] Processing DeleteTablet for tablet 7266eb74523740e5a4c79c830b0aa0e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:09 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:59132
I20260809 03:39:09.902675 28977 raft_consensus.cc:2258] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:39:09.903453 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 28292
I20260809 03:39:09.903473 28979 tablet_replica.cc:330] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89: stopping tablet replica
I20260809 03:39:09.903769 28979 raft_consensus.cc:2229] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:39:09.904253 28979 raft_consensus.cc:2258] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:39:09.904776 28977 ts_tablet_manager.cc:1905] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:39:09.909094 28977 ts_tablet_manager.cc:1918] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:39:09.909318 28977 log.cc:1196] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/7266eb74523740e5a4c79c830b0aa0e5
I20260809 03:39:09.909539 28979 ts_tablet_manager.cc:1905] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:39:09.909982 28977 ts_tablet_manager.cc:1939] T 7266eb74523740e5a4c79c830b0aa0e5 P 645e10afc2b547e79e667606cf6f68d4: Deleting consensus metadata
I20260809 03:39:09.911921 28176 catalog_manager.cc:4841] TS 645e10afc2b547e79e667606cf6f68d4 (127.1.224.66:42243): tablet 7266eb74523740e5a4c79c830b0aa0e5 (table test-table [id=9a8eaf8ebb0740aaad1dda25213b97b8]) successfully deleted
I20260809 03:39:09.912436 28979 ts_tablet_manager.cc:1918] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:39:09.912575 28979 log.cc:1196] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/7266eb74523740e5a4c79c830b0aa0e5
I20260809 03:39:09.912909 28979 ts_tablet_manager.cc:1939] T 7266eb74523740e5a4c79c830b0aa0e5 P 73f803b8214e480ead97b5b4f2c3ba89: Deleting consensus metadata
I20260809 03:39:09.914299 28178 catalog_manager.cc:4841] TS 73f803b8214e480ead97b5b4f2c3ba89 (127.1.224.67:43861): tablet 7266eb74523740e5a4c79c830b0aa0e5 (table test-table [id=9a8eaf8ebb0740aaad1dda25213b97b8]) successfully deleted
W20260809 03:39:09.925585 28179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:33549 (error 108)
I20260809 03:39:09.925688 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 28510
I20260809 03:39:09.937363 28984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:09.936861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:33549 (local address 127.0.0.1:59148)
0809 03:39:09.937156 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:09.937248 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:33549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":97,"threads_started":1}
W20260809 03:39:09.937682 28179 catalog_manager.cc:4584] TS 0c59c136c7c14e6db235fc36b4864be1 (127.1.224.65:33549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7266eb74523740e5a4c79c830b0aa0e5: Network error: Client connection negotiation failed: client connection to 127.1.224.65:33549: connect: Connection refused (error 111)
I20260809 03:39:09.943262 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 28711
I20260809 03:39:09.958180 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 28152
2026-08-09T03:39:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (3032 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T03:39:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:09Z Disabled control of system clock
I20260809 03:39:09.998992 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:32795
--webserver_interface=127.1.224.126
--webserver_port=0
--builtin_ntp_servers=127.1.224.84:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:32795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:39:10.117923 28992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:10.118233 28992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:10.118271 28992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:10.122423 28992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:10.122505 28992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:10.122521 28992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:10.122538 28992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:10.122555 28992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:10.127575 28992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:32795
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:32795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:10.128932 28992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:10.130872 28992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:10.139883 28998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:10.139883 29004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:10.139933 28997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:10.140228 28992 server_base.cc:1034] running on GCE node
I20260809 03:39:10.140986 28992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:10.142201 28992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:10.143429 28992 hybrid_clock.cc:648] HybridClock initialized: now 1786246750143402 us; error 48 us; skew 500 ppm
I20260809 03:39:10.145622 28992 webserver.cc:460] Webserver started at http://127.1.224.126:39457/ using document root <none> and password file <none>
I20260809 03:39:10.146255 28992 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:10.146350 28992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:10.146672 28992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:10.148988 28992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "65f03fecef834b33a25ed506b671a9a5"
format_stamp: "Formatted at 2026-08-09 03:39:10 on dist-test-slave-jq4t"
server_key: "5c57bb2efcaf3a865f4f988e09aa44b5"
server_key_iv: "dc7e30ae734e3da3fe124bf0081d0759"
server_key_version: "encryptionkey@0"
I20260809 03:39:10.149596 28992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "65f03fecef834b33a25ed506b671a9a5"
format_stamp: "Formatted at 2026-08-09 03:39:10 on dist-test-slave-jq4t"
server_key: "5c57bb2efcaf3a865f4f988e09aa44b5"
server_key_iv: "dc7e30ae734e3da3fe124bf0081d0759"
server_key_version: "encryptionkey@0"
I20260809 03:39:10.153796 28992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 03:39:10.156802 29012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:10.158246 28992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 03:39:10.158437 28992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "65f03fecef834b33a25ed506b671a9a5"
format_stamp: "Formatted at 2026-08-09 03:39:10 on dist-test-slave-jq4t"
server_key: "5c57bb2efcaf3a865f4f988e09aa44b5"
server_key_iv: "dc7e30ae734e3da3fe124bf0081d0759"
server_key_version: "encryptionkey@0"
I20260809 03:39:10.158584 28992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:10.186524 28992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:10.187410 28992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:10.187649 28992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:10.195678 28992 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:32795
I20260809 03:39:10.195753 29087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:32795 every 8 connection(s)
I20260809 03:39:10.196906 28992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:10.200366 29088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:10.206190 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 28992
I20260809 03:39:10.206362 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:10.206594 29088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5: Bootstrap starting.
I20260809 03:39:10.206676 1921 external_mini_cluster.cc:1428] Setting key 767d9104d68510ac7565b2a423806e9f
I20260809 03:39:10.209396 29088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:10.210196 29088 log.cc:824] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:10.212181 29088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5: No bootstrap required, opened a new log
I20260809 03:39:10.215850 29088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f03fecef834b33a25ed506b671a9a5" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 32795 } }
I20260809 03:39:10.216104 29088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:10.216190 29088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65f03fecef834b33a25ed506b671a9a5, State: Initialized, Role: FOLLOWER
I20260809 03:39:10.216310 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.208194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.1.224.126:32795)
0809 03:39:10.208939 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:10.208955 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:10.208992 (+ 37us) server_negotiation.cc:407] Connection header received
0809 03:39:10.209705 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:10.209725 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:10.210139 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:10.210478 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:10.210943 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.211929 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.212951 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.213428 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.213603 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.214494 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:10.214499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:10.214512 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:10.214910 (+ 398us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:10.214967 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:10.214986 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:10.215091 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:10.215211 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 03:39:10.215493 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":144,"threads_started":1}
I20260809 03:39:10.216745 29088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f03fecef834b33a25ed506b671a9a5" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 32795 } }
I20260809 03:39:10.216921 29088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:10.216981 29088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:10.217082 29088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:10.218140 29088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f03fecef834b33a25ed506b671a9a5" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 32795 } }
I20260809 03:39:10.218523 29088 leader_election.cc:302] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65f03fecef834b33a25ed506b671a9a5; no voters:
I20260809 03:39:10.218870 29088 leader_election.cc:288] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:10.219017 29094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:10.219255 29094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [term 1 LEADER]: Becoming Leader. State: Replica: 65f03fecef834b33a25ed506b671a9a5, State: Running, Role: LEADER
I20260809 03:39:10.219547 29094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f03fecef834b33a25ed506b671a9a5" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 32795 } }
I20260809 03:39:10.220088 29088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:10.221341 29100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f03fecef834b33a25ed506b671a9a5. Latest consensus state: current_term: 1 leader_uuid: "65f03fecef834b33a25ed506b671a9a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f03fecef834b33a25ed506b671a9a5" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 32795 } } }
I20260809 03:39:10.221357 29099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65f03fecef834b33a25ed506b671a9a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f03fecef834b33a25ed506b671a9a5" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 32795 } } }
I20260809 03:39:10.221542 29099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:10.221825 29100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:10.221967 29108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:10.225318 29108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:10.231292 29108 catalog_manager.cc:1347] Generated new cluster ID: c4225d75071443a4a8f1d583264f650c
I20260809 03:39:10.231366 29108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:10.244168 29108 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:10.245287 29108 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:10.255838 29108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5: Generated new TSK 0
I20260809 03:39:10.257014 29108 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:10.266633 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:32795
--builtin_ntp_servers=127.1.224.84:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:39:10.398888 29122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:10.399422 29122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:10.399492 29122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:10.406497 29122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:10.406575 29122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:10.406635 29122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:39:10.415170 29122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:32795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:10.416528 29122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:10.418167 29122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:10.428753 29131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:10.428745 29135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:10.428745 29133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:10.429371 29122 server_base.cc:1034] running on GCE node
I20260809 03:39:10.429809 29122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:10.430483 29122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:10.431697 29122 hybrid_clock.cc:648] HybridClock initialized: now 1786246750431683 us; error 33 us; skew 500 ppm
I20260809 03:39:10.433905 29122 webserver.cc:460] Webserver started at http://127.1.224.65:45769/ using document root <none> and password file <none>
I20260809 03:39:10.434587 29122 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:10.434685 29122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:10.434952 29122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:10.437299 29122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "009de27b5d1343afb252e1c6203e879f"
format_stamp: "Formatted at 2026-08-09 03:39:10 on dist-test-slave-jq4t"
server_key: "ccde498476465e595f5355d4887fc310"
server_key_iv: "854b139d2e6ba9a0ddefc1068550f0c3"
server_key_version: "encryptionkey@0"
I20260809 03:39:10.437917 29122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "009de27b5d1343afb252e1c6203e879f"
format_stamp: "Formatted at 2026-08-09 03:39:10 on dist-test-slave-jq4t"
server_key: "ccde498476465e595f5355d4887fc310"
server_key_iv: "854b139d2e6ba9a0ddefc1068550f0c3"
server_key_version: "encryptionkey@0"
I20260809 03:39:10.441813 29122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:39:10.444382 29144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:10.445888 29122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:39:10.446060 29122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "009de27b5d1343afb252e1c6203e879f"
format_stamp: "Formatted at 2026-08-09 03:39:10 on dist-test-slave-jq4t"
server_key: "ccde498476465e595f5355d4887fc310"
server_key_iv: "854b139d2e6ba9a0ddefc1068550f0c3"
server_key_version: "encryptionkey@0"
I20260809 03:39:10.446194 29122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:10.461452 29122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:10.462258 29122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:10.462487 29122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:10.463238 29122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:10.464416 29122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:10.464529 29122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:10.464614 29122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:10.464660 29122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:10.474438 29122 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:36091
I20260809 03:39:10.474462 29303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:36091 every 8 connection(s)
I20260809 03:39:10.475453 29122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:39:10.485189 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.477487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:58977 (local address 127.1.224.126:32795)
0809 03:39:10.477670 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:10.477676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:10.478811 (+ 1135us) server_negotiation.cc:407] Connection header received
0809 03:39:10.480186 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:10.480188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:10.480236 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:10.480312 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:10.481302 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.481853 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.483027 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.483265 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.483358 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.484677 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:10.484681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:10.484683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:10.484698 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:10.484722 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:10.484730 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:10.484800 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:10.484963 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 03:39:10.485027 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 03:39:10.485487 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 29122
I20260809 03:39:10.485627 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:39:10.485946 1921 external_mini_cluster.cc:1428] Setting key e6f463ae5c6c747375797ffea255e93a
I20260809 03:39:10.486280 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.477930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:32795 (local address 127.1.224.65:58977)
0809 03:39:10.478648 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:10.478692 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:10.479989 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:10.480499 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:10.480506 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:10.480592 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:10.481167 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.481181 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.481963 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.481966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.482875 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.482885 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.483432 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.483435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.483587 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.484300 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:39:10.484301 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:10.484346 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:39:10.484429 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:10.484814 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:10.484823 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:10.484828 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:39:10.484940 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 03:39:10.485194 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":145,"threads_started":1}
I20260809 03:39:10.487712 29304 heartbeater.cc:347] Connected to a master server at 127.1.224.126:32795
I20260809 03:39:10.487985 29304 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:10.488286 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:32795
--builtin_ntp_servers=127.1.224.84:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:39:10.488706 29304 heartbeater.cc:511] Master 127.1.224.126:32795 requested a full tablet report, sending...
I20260809 03:39:10.490497 29037 ts_manager.cc:194] Registered new tserver with Master: 009de27b5d1343afb252e1c6203e879f (127.1.224.65:36091)
I20260809 03:39:10.491990 29037 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.224.65:58977
W20260809 03:39:10.607091 29307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:10.607378 29307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:10.607416 29307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:10.611377 29307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:10.611440 29307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:10.611493 29307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:39:10.616135 29307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:32795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:10.617333 29307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:10.618825 29307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:10.628871 29315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:10.628858 29312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:10.628939 29307 server_base.cc:1034] running on GCE node
W20260809 03:39:10.628861 29319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:10.629631 29307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:10.630260 29307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:10.631420 29307 hybrid_clock.cc:648] HybridClock initialized: now 1786246750631401 us; error 31 us; skew 500 ppm
I20260809 03:39:10.633358 29307 webserver.cc:460] Webserver started at http://127.1.224.66:36313/ using document root <none> and password file <none>
I20260809 03:39:10.633946 29307 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:10.634014 29307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:10.634212 29307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:10.636896 29307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "031137f29700458689cf7ea490015935"
format_stamp: "Formatted at 2026-08-09 03:39:10 on dist-test-slave-jq4t"
server_key: "3d43acf2276d69b5780836236538e93a"
server_key_iv: "ba6a666cd1c49f3fd62ceedd6145d6dd"
server_key_version: "encryptionkey@0"
I20260809 03:39:10.637452 29307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "031137f29700458689cf7ea490015935"
format_stamp: "Formatted at 2026-08-09 03:39:10 on dist-test-slave-jq4t"
server_key: "3d43acf2276d69b5780836236538e93a"
server_key_iv: "ba6a666cd1c49f3fd62ceedd6145d6dd"
server_key_version: "encryptionkey@0"
I20260809 03:39:10.641408 29307 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:39:10.644042 29328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:10.645433 29307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:10.645555 29307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "031137f29700458689cf7ea490015935"
format_stamp: "Formatted at 2026-08-09 03:39:10 on dist-test-slave-jq4t"
server_key: "3d43acf2276d69b5780836236538e93a"
server_key_iv: "ba6a666cd1c49f3fd62ceedd6145d6dd"
server_key_version: "encryptionkey@0"
I20260809 03:39:10.645658 29307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:10.679826 29307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:10.680637 29307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:10.680814 29307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:10.681418 29307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:10.682449 29307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:10.682520 29307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:10.682566 29307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:10.682588 29307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:10.692334 29307 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:35037
I20260809 03:39:10.692369 29500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:35037 every 8 connection(s)
I20260809 03:39:10.693410 29307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:39:10.697216 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 29307
I20260809 03:39:10.697369 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:39:10.697737 1921 external_mini_cluster.cc:1428] Setting key 176986d80d47439f52221c094f12c310
I20260809 03:39:10.700682 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:32795
--builtin_ntp_servers=127.1.224.84:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:39:10.705384 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.695510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:54651 (local address 127.1.224.126:32795)
0809 03:39:10.695668 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:10.695672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:10.697310 (+ 1638us) server_negotiation.cc:407] Connection header received
0809 03:39:10.699237 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:10.699241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:10.699292 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:10.699385 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:10.700647 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.701280 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.703008 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.703389 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.703529 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.704747 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:10.704754 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:10.704760 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:10.704782 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:10.704811 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:10.704825 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:10.704906 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:10.705075 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 03:39:10.705155 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:39:10.706235 29508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.696048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:32795 (local address 127.1.224.66:54651)
0809 03:39:10.697105 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:10.697174 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:10.699015 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:10.699548 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:10.699555 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:10.699690 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:10.700431 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.700469 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.701424 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.701431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.702877 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.702889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.703556 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.703559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.703711 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.704394 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:39:10.704395 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:10.704439 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:39:10.704579 (+ 140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:10.704922 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:10.704930 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:10.704935 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:39:10.705032 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 03:39:10.705287 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":194,"threads_started":1}
I20260809 03:39:10.707562 29502 heartbeater.cc:347] Connected to a master server at 127.1.224.126:32795
I20260809 03:39:10.707819 29502 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:10.708407 29502 heartbeater.cc:511] Master 127.1.224.126:32795 requested a full tablet report, sending...
I20260809 03:39:10.709599 29037 ts_manager.cc:194] Registered new tserver with Master: 031137f29700458689cf7ea490015935 (127.1.224.66:35037)
I20260809 03:39:10.710218 29037 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.224.66:54651
W20260809 03:39:10.825881 29509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:10.826152 29509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:10.826218 29509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:10.830413 29509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:10.830476 29509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:10.830581 29509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:39:10.835635 29509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:32795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:10.836887 29509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:10.838356 29509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:10.848661 29517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:10.848774 29509 server_base.cc:1034] running on GCE node
W20260809 03:39:10.848582 29520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:10.848591 29516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:10.849414 29509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:10.850171 29509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:10.851351 29509 hybrid_clock.cc:648] HybridClock initialized: now 1786246750851327 us; error 33 us; skew 500 ppm
I20260809 03:39:10.853281 29509 webserver.cc:460] Webserver started at http://127.1.224.67:32931/ using document root <none> and password file <none>
I20260809 03:39:10.853904 29509 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:10.854002 29509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:10.854240 29509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:10.856376 29509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294"
format_stamp: "Formatted at 2026-08-09 03:39:10 on dist-test-slave-jq4t"
server_key: "c2dfba314495ee9de32bd346c4c31441"
server_key_iv: "a31c944ae561fd6d97065c73b44b1bbb"
server_key_version: "encryptionkey@0"
I20260809 03:39:10.857043 29509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294"
format_stamp: "Formatted at 2026-08-09 03:39:10 on dist-test-slave-jq4t"
server_key: "c2dfba314495ee9de32bd346c4c31441"
server_key_iv: "a31c944ae561fd6d97065c73b44b1bbb"
server_key_version: "encryptionkey@0"
I20260809 03:39:10.861351 29509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:39:10.864024 29532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:10.865410 29509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:10.865566 29509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294"
format_stamp: "Formatted at 2026-08-09 03:39:10 on dist-test-slave-jq4t"
server_key: "c2dfba314495ee9de32bd346c4c31441"
server_key_iv: "a31c944ae561fd6d97065c73b44b1bbb"
server_key_version: "encryptionkey@0"
I20260809 03:39:10.865694 29509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:10.889503 29509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:10.890348 29509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:10.890587 29509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:10.891285 29509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:10.892344 29509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:10.892418 29509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:10.892534 29509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:10.892576 29509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:10.901993 29509 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:36161
I20260809 03:39:10.902019 29711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:36161 every 8 connection(s)
I20260809 03:39:10.903105 29509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:39:10.910627 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 29509
I20260809 03:39:10.910760 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:39:10.911136 1921 external_mini_cluster.cc:1428] Setting key e8f5901b6ebfc4b7c901f96ceee93e6b
I20260809 03:39:10.913466 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.905372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:37787 (local address 127.1.224.126:32795)
0809 03:39:10.905540 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:10.905545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:10.906719 (+ 1174us) server_negotiation.cc:407] Connection header received
0809 03:39:10.908106 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:10.908110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:10.908161 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:10.908241 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:10.909183 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.909720 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.910988 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.911315 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.911433 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.912991 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:10.912995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:10.912997 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:10.913010 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:10.913030 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:10.913038 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:10.913094 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:10.913291 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 03:39:10.913346 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:39:10.914564 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.905851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:32795 (local address 127.1.224.67:37787)
0809 03:39:10.906558 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:10.906601 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:10.907912 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:10.908415 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:10.908422 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:10.908536 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:10.909051 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.909063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.909829 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.909831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.910817 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.910839 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.911542 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.911548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.911775 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.912647 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:39:10.912650 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:10.912711 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:39:10.912806 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:10.913117 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:10.913134 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:10.913140 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:39:10.913278 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 03:39:10.913700 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":129,"threads_started":1}
I20260809 03:39:10.915666 29713 heartbeater.cc:347] Connected to a master server at 127.1.224.126:32795
I20260809 03:39:10.915925 29713 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:10.916530 29713 heartbeater.cc:511] Master 127.1.224.126:32795 requested a full tablet report, sending...
I20260809 03:39:10.917587 29037 ts_manager.cc:194] Registered new tserver with Master: ca5c93e2d4e5495fbefb2fcf02e7d294 (127.1.224.67:36161)
I20260809 03:39:10.918203 29037 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.224.67:37787
I20260809 03:39:10.925572 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:39:10.931181 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.927596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46402 (local address 127.1.224.126:32795)
0809 03:39:10.927718 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:10.927727 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:10.927865 (+ 138us) server_negotiation.cc:407] Connection header received
0809 03:39:10.927967 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:10.927969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:10.928014 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:10.928085 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:10.928345 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.928940 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.929750 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.929956 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.930036 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.930824 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:10.930836 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:10.930838 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:10.930849 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:10.930864 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:10.930871 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:10.930925 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:10.931020 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 03:39:10.931071 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:39:10.934090 29037 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:39:10.937335 29037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:39:10.953689 29734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.947637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59160 (local address 127.1.224.67:36161)
0809 03:39:10.948367 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:10.948380 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:10.948404 (+ 24us) server_negotiation.cc:407] Connection header received
0809 03:39:10.948518 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:10.948524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:10.948714 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:10.948871 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:10.949128 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.950306 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.951950 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.952837 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.952964 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.953171 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:10.953376 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 03:39:10.953500 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":114,"threads_started":1}
I20260809 03:39:10.953682 29735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.947773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:36091 (local address 127.0.0.1:58616)
0809 03:39:10.948020 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:10.948032 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:10.948101 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:10.949234 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:10.949237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:10.949255 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:10.949369 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.949377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.950824 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.950841 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.951986 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.951995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.953240 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.953244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.953357 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.953384 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 03:39:10.953458 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 03:39:10.953525 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":93,"threads_started":1}
I20260809 03:39:10.953687 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.947433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:36161 (local address 127.0.0.1:59160)
0809 03:39:10.947728 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:10.947745 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:10.947839 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:10.948868 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:10.948871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:10.948891 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:10.948993 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.948998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.950441 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.950444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.951741 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.951753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.952978 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.952984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.953132 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.953211 (+ 79us) client_negotiation.cc:769] Sending connection context
0809 03:39:10.953330 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 03:39:10.953469 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":71,"thread_start_us":128,"threads_started":1}
I20260809 03:39:10.953976 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.946680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:35037 (local address 127.0.0.1:54326)
0809 03:39:10.947050 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:10.947110 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:10.947276 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:10.948040 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:10.948049 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:10.948078 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:10.948252 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.948260 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.949813 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.949818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.952117 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.952133 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.953056 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.953060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.953136 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.953211 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 03:39:10.953306 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 03:39:10.953469 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":907,"thread_start_us":80,"threads_started":1}
I20260809 03:39:10.954491 29736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.948420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58616 (local address 127.1.224.65:36091)
0809 03:39:10.948744 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:10.948751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:10.948776 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:39:10.948855 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:10.948860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:10.949045 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:10.949274 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:10.949467 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.950699 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.952112 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.952885 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.952985 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.953172 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:10.954205 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0809 03:39:10.954317 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":114,"threads_started":1}
I20260809 03:39:10.954594 29728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.946791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54326 (local address 127.1.224.66:35037)
0809 03:39:10.947469 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:10.947486 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:10.947526 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:39:10.947619 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:10.947625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:10.947871 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:10.948070 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:10.948389 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.949671 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.952278 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.952920 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.953618 (+ 698us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.954229 (+ 611us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:10.954292 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 03:39:10.954402 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":188,"threads_started":1}
I20260809 03:39:10.957121 29403 tablet_service.cc:1467] Processing CreateTablet for tablet f61b8298f48742bc83a14164242eabbd (DEFAULT_TABLE table=test-table [id=9ecea6ea48f84ccaac20e35aa700207e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:10.957121 29607 tablet_service.cc:1467] Processing CreateTablet for tablet f61b8298f48742bc83a14164242eabbd (DEFAULT_TABLE table=test-table [id=9ecea6ea48f84ccaac20e35aa700207e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:10.958268 29403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f61b8298f48742bc83a14164242eabbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:10.958271 29607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f61b8298f48742bc83a14164242eabbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:10.959647 29220 tablet_service.cc:1467] Processing CreateTablet for tablet f61b8298f48742bc83a14164242eabbd (DEFAULT_TABLE table=test-table [id=9ecea6ea48f84ccaac20e35aa700207e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:10.960855 29220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f61b8298f48742bc83a14164242eabbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:10.965057 29737 tablet_bootstrap.cc:492] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294: Bootstrap starting.
I20260809 03:39:10.967272 29737 tablet_bootstrap.cc:654] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:10.967710 29740 tablet_bootstrap.cc:492] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f: Bootstrap starting.
I20260809 03:39:10.967805 29738 tablet_bootstrap.cc:492] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935: Bootstrap starting.
I20260809 03:39:10.968556 29737 log.cc:824] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:10.970112 29738 tablet_bootstrap.cc:654] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:10.970198 29740 tablet_bootstrap.cc:654] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:10.970901 29737 tablet_bootstrap.cc:492] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294: No bootstrap required, opened a new log
I20260809 03:39:10.970968 29738 log.cc:824] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:10.971078 29740 log.cc:824] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:10.971117 29737 ts_tablet_manager.cc:1397] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:39:10.972908 29738 tablet_bootstrap.cc:492] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935: No bootstrap required, opened a new log
I20260809 03:39:10.973109 29738 ts_tablet_manager.cc:1397] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:39:10.973850 29740 tablet_bootstrap.cc:492] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f: No bootstrap required, opened a new log
I20260809 03:39:10.974045 29740 ts_tablet_manager.cc:1397] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:39:10.975432 29737 raft_consensus.cc:348] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031137f29700458689cf7ea490015935" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35037 } } peers { permanent_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36161 } } peers { permanent_uuid: "009de27b5d1343afb252e1c6203e879f" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 36091 } }
I20260809 03:39:10.975720 29737 raft_consensus.cc:374] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:10.975786 29737 raft_consensus.cc:729] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5c93e2d4e5495fbefb2fcf02e7d294, State: Initialized, Role: FOLLOWER
I20260809 03:39:10.976293 29737 consensus_queue.cc:260] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031137f29700458689cf7ea490015935" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35037 } } peers { permanent_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36161 } } peers { permanent_uuid: "009de27b5d1343afb252e1c6203e879f" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 36091 } }
I20260809 03:39:10.976735 29738 raft_consensus.cc:348] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031137f29700458689cf7ea490015935" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35037 } } peers { permanent_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36161 } } peers { permanent_uuid: "009de27b5d1343afb252e1c6203e879f" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 36091 } }
I20260809 03:39:10.977011 29738 raft_consensus.cc:374] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:10.977077 29738 raft_consensus.cc:729] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 031137f29700458689cf7ea490015935, State: Initialized, Role: FOLLOWER
I20260809 03:39:10.977324 29713 heartbeater.cc:503] Master 127.1.224.126:32795 was elected leader, sending a full tablet report...
I20260809 03:39:10.977519 29737 ts_tablet_manager.cc:1428] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 03:39:10.978016 29738 consensus_queue.cc:260] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031137f29700458689cf7ea490015935" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35037 } } peers { permanent_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36161 } } peers { permanent_uuid: "009de27b5d1343afb252e1c6203e879f" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 36091 } }
I20260809 03:39:10.978093 29740 raft_consensus.cc:348] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031137f29700458689cf7ea490015935" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35037 } } peers { permanent_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36161 } } peers { permanent_uuid: "009de27b5d1343afb252e1c6203e879f" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 36091 } }
I20260809 03:39:10.978366 29740 raft_consensus.cc:374] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:10.978436 29740 raft_consensus.cc:729] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 009de27b5d1343afb252e1c6203e879f, State: Initialized, Role: FOLLOWER
I20260809 03:39:10.978827 29502 heartbeater.cc:503] Master 127.1.224.126:32795 was elected leader, sending a full tablet report...
I20260809 03:39:10.978874 29740 consensus_queue.cc:260] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031137f29700458689cf7ea490015935" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35037 } } peers { permanent_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36161 } } peers { permanent_uuid: "009de27b5d1343afb252e1c6203e879f" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 36091 } }
I20260809 03:39:10.979535 29304 heartbeater.cc:503] Master 127.1.224.126:32795 was elected leader, sending a full tablet report...
I20260809 03:39:10.980093 29738 ts_tablet_manager.cc:1428] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:39:10.980881 29740 ts_tablet_manager.cc:1428] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 03:39:10.983703 29746 raft_consensus.cc:482] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:39:10.983886 29746 raft_consensus.cc:504] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031137f29700458689cf7ea490015935" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35037 } } peers { permanent_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36161 } } peers { permanent_uuid: "009de27b5d1343afb252e1c6203e879f" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 36091 } }
I20260809 03:39:10.985014 29746 leader_election.cc:288] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 031137f29700458689cf7ea490015935 (127.1.224.66:35037), 009de27b5d1343afb252e1c6203e879f (127.1.224.65:36091)
I20260809 03:39:10.989737 29736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.985578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:51347 (local address 127.1.224.65:36091)
0809 03:39:10.985899 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:10.985902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:10.985913 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:39:10.986029 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:10.986031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:10.986075 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:10.986152 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:10.986348 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.987060 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.988409 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.989030 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.989157 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.989210 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:10.989534 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 03:39:10.989606 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248}
I20260809 03:39:10.989737 29728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.985303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:43303 (local address 127.1.224.66:35037)
0809 03:39:10.985424 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:10.985429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:10.985442 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:39:10.985580 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:10.985587 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:10.985651 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:10.985732 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:10.986037 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.986867 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.988396 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.989030 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.989157 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.989209 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:10.989534 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 03:39:10.989609 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:39:10.989754 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.985234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:35037 (local address 127.1.224.67:43303)
0809 03:39:10.985355 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:10.985373 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:10.985466 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:10.985774 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:10.985777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:10.985797 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:10.985928 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.985935 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.986969 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.986972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.988251 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.988264 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.989153 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.989157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.989341 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.989422 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 03:39:10.989514 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 03:39:10.989597 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":102}
I20260809 03:39:10.989778 29751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:10.985478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:36091 (local address 127.1.224.67:51347)
0809 03:39:10.985840 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:10.985854 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:10.985930 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:10.986158 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:10.986160 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:10.986172 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:10.986252 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.986256 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.987154 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.987157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.988288 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:10.988299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:10.989157 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:10.989159 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:10.989384 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:10.989435 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 03:39:10.989506 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 03:39:10.989598 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":110,"thread_start_us":145,"threads_started":1}
I20260809 03:39:10.990448 29436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61b8298f48742bc83a14164242eabbd" candidate_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "031137f29700458689cf7ea490015935" is_pre_election: true
I20260809 03:39:10.990532 29248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61b8298f48742bc83a14164242eabbd" candidate_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "009de27b5d1343afb252e1c6203e879f" is_pre_election: true
I20260809 03:39:10.990775 29436 raft_consensus.cc:2454] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5c93e2d4e5495fbefb2fcf02e7d294 in term 0.
I20260809 03:39:10.990819 29248 raft_consensus.cc:2454] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5c93e2d4e5495fbefb2fcf02e7d294 in term 0.
I20260809 03:39:10.991319 29534 leader_election.cc:302] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031137f29700458689cf7ea490015935, ca5c93e2d4e5495fbefb2fcf02e7d294; no voters:
I20260809 03:39:10.991629 29746 raft_consensus.cc:2781] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:39:10.991729 29746 raft_consensus.cc:482] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:39:10.991787 29746 raft_consensus.cc:3037] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:10.992949 29746 raft_consensus.cc:504] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031137f29700458689cf7ea490015935" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35037 } } peers { permanent_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36161 } } peers { permanent_uuid: "009de27b5d1343afb252e1c6203e879f" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 36091 } }
I20260809 03:39:10.993320 29746 leader_election.cc:288] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [CANDIDATE]: Term 1 election: Requested vote from peers 031137f29700458689cf7ea490015935 (127.1.224.66:35037), 009de27b5d1343afb252e1c6203e879f (127.1.224.65:36091)
I20260809 03:39:10.993769 29248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61b8298f48742bc83a14164242eabbd" candidate_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "009de27b5d1343afb252e1c6203e879f"
I20260809 03:39:10.993786 29436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f61b8298f48742bc83a14164242eabbd" candidate_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "031137f29700458689cf7ea490015935"
I20260809 03:39:10.993906 29248 raft_consensus.cc:3037] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:10.993937 29436 raft_consensus.cc:3037] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:10.995286 29248 raft_consensus.cc:2454] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5c93e2d4e5495fbefb2fcf02e7d294 in term 1.
I20260809 03:39:10.995357 29436 raft_consensus.cc:2454] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5c93e2d4e5495fbefb2fcf02e7d294 in term 1.
I20260809 03:39:10.995608 29534 leader_election.cc:302] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 009de27b5d1343afb252e1c6203e879f, ca5c93e2d4e5495fbefb2fcf02e7d294; no voters:
I20260809 03:39:10.995844 29746 raft_consensus.cc:2781] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:10.996095 29746 raft_consensus.cc:686] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 1 LEADER]: Becoming Leader. State: Replica: ca5c93e2d4e5495fbefb2fcf02e7d294, State: Running, Role: LEADER
I20260809 03:39:10.996477 29746 consensus_queue.cc:237] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031137f29700458689cf7ea490015935" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35037 } } peers { permanent_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36161 } } peers { permanent_uuid: "009de27b5d1343afb252e1c6203e879f" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 36091 } }
I20260809 03:39:10.999821 29037 catalog_manager.cc:5495] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca5c93e2d4e5495fbefb2fcf02e7d294 (127.1.224.67). New cstate: current_term: 1 leader_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031137f29700458689cf7ea490015935" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5c93e2d4e5495fbefb2fcf02e7d294" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 36161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "009de27b5d1343afb252e1c6203e879f" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 36091 } health_report { overall_health: UNKNOWN } } }
I20260809 03:39:11.015947 29734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:11.012560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.1.224.67:36161)
0809 03:39:11.012697 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:11.012701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:11.012714 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:39:11.012792 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:11.012794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:11.012849 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:11.012939 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:11.013154 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:11.013661 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:11.014496 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:11.014720 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:11.014803 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:11.015075 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:11.015534 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:11.015636 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:11.015682 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 03:39:11.015732 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:39:11.022930 29248 raft_consensus.cc:1261] T f61b8298f48742bc83a14164242eabbd P 009de27b5d1343afb252e1c6203e879f [term 1 FOLLOWER]: Refusing update from remote peer ca5c93e2d4e5495fbefb2fcf02e7d294: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:11.022935 29436 raft_consensus.cc:1261] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935 [term 1 FOLLOWER]: Refusing update from remote peer ca5c93e2d4e5495fbefb2fcf02e7d294: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:11.023638 29746 consensus_queue.cc:1035] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "009de27b5d1343afb252e1c6203e879f" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 36091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:11.023911 29752 consensus_queue.cc:1035] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "031137f29700458689cf7ea490015935" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 35037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:11.031575 29758 mvcc.cc:205] Tried to move back new op lower bound from 7316466692181782528 to 7316466692084772864. Current Snapshot: MvccSnapshot[applied={T|T < 7316466692181782528}]
I20260809 03:39:11.042286 29036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:39:11.042520 29036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:39:11.045629 29036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65f03fecef834b33a25ed506b671a9a5: Sending DeleteTablet for 3 replicas of tablet f61b8298f48742bc83a14164242eabbd
I20260809 03:39:11.046579 29403 tablet_service.cc:1514] Processing DeleteTablet for tablet f61b8298f48742bc83a14164242eabbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:11 UTC) from {username='slave'} at 127.0.0.1:54326
I20260809 03:39:11.046752 29220 tablet_service.cc:1514] Processing DeleteTablet for tablet f61b8298f48742bc83a14164242eabbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:11 UTC) from {username='slave'} at 127.0.0.1:58616
I20260809 03:39:11.046868 29607 tablet_service.cc:1514] Processing DeleteTablet for tablet f61b8298f48742bc83a14164242eabbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:11 UTC) from {username='slave'} at 127.0.0.1:59160
I20260809 03:39:11.047348 29771 tablet_replica.cc:330] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294: stopping tablet replica
I20260809 03:39:11.047732 29771 raft_consensus.cc:2229] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:39:11.047835 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 29122
I20260809 03:39:11.056372 29768 tablet_replica.cc:330] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935: stopping tablet replica
I20260809 03:39:11.056713 29771 raft_consensus.cc:2258] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:39:11.056859 29768 raft_consensus.cc:2229] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:39:11.057247 29768 raft_consensus.cc:2258] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:39:11.058673 29771 ts_tablet_manager.cc:1905] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:39:11.058871 29768 ts_tablet_manager.cc:1905] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:39:11.061527 29771 ts_tablet_manager.cc:1918] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:39:11.061635 29771 log.cc:1196] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/f61b8298f48742bc83a14164242eabbd
I20260809 03:39:11.061954 29771 ts_tablet_manager.cc:1939] T f61b8298f48742bc83a14164242eabbd P ca5c93e2d4e5495fbefb2fcf02e7d294: Deleting consensus metadata
I20260809 03:39:11.062342 29768 ts_tablet_manager.cc:1918] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:39:11.062542 29768 log.cc:1196] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/f61b8298f48742bc83a14164242eabbd
I20260809 03:39:11.062891 29768 ts_tablet_manager.cc:1939] T f61b8298f48742bc83a14164242eabbd P 031137f29700458689cf7ea490015935: Deleting consensus metadata
I20260809 03:39:11.063498 29013 catalog_manager.cc:4841] TS ca5c93e2d4e5495fbefb2fcf02e7d294 (127.1.224.67:36161): tablet f61b8298f48742bc83a14164242eabbd (table test-table [id=9ecea6ea48f84ccaac20e35aa700207e]) successfully deleted
I20260809 03:39:11.064193 29013 catalog_manager.cc:4841] TS 031137f29700458689cf7ea490015935 (127.1.224.66:35037): tablet f61b8298f48742bc83a14164242eabbd (table test-table [id=9ecea6ea48f84ccaac20e35aa700207e]) successfully deleted
W20260809 03:39:11.064872 29013 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:36091 (error 108)
I20260809 03:39:11.064999 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 29307
I20260809 03:39:11.075363 29735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:11.075010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:36091 (local address 127.0.0.1:58626)
0809 03:39:11.075169 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:11.075266 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:36091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 03:39:11.075702 29013 catalog_manager.cc:4584] TS 009de27b5d1343afb252e1c6203e879f (127.1.224.65:36091): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f61b8298f48742bc83a14164242eabbd: Network error: Client connection negotiation failed: client connection to 127.1.224.65:36091: connect: Connection refused (error 111)
I20260809 03:39:11.083194 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 29509
I20260809 03:39:11.100931 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 28992
2026-08-09T03:39:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1141 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:11 dist-test-slave-jq4t krb5kdc[29778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:11 dist-test-slave-jq4t krb5kdc[29778](info): set up 2 sockets
Aug 09 03:39:11 dist-test-slave-jq4t krb5kdc[29778](info): commencing operation
krb5kdc: starting...
W20260809 03:39:12.363998 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:12 dist-test-slave-jq4t krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:39:12.499722 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:45437
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:45437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:12.621137 29801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:12.621506 29801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:12.621582 29801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:12.625844 29801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:12.625952 29801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:12.625990 29801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:12.626042 29801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:12.626101 29801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:12.632085 29801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:45437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:45437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:12.633406 29801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:12.634543 29801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:12.643755 29817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:12.643922 29807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:12.643783 29809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:12.644053 29801 server_base.cc:1034] running on GCE node
I20260809 03:39:12.644764 29801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:12.645967 29801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:12.647118 29801 hybrid_clock.cc:648] HybridClock initialized: now 1786246752647101 us; error 32 us; skew 500 ppm
Aug 09 03:39:12 dist-test-slave-jq4t krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:12.650148 29801 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:39:12.651121 29801 webserver.cc:460] Webserver started at http://127.1.224.126:36943/ using document root <none> and password file <none>
I20260809 03:39:12.651738 29801 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:12.651824 29801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:12.652065 29801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:12.654385 29801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "6a45654d2bb445198a2f4897d9257b74"
format_stamp: "Formatted at 2026-08-09 03:39:12 on dist-test-slave-jq4t"
server_key: "1930ce82caf05ae8eb135c0236558b8d"
server_key_iv: "56705a922157b004640786c458dfea7f"
server_key_version: "encryptionkey@0"
I20260809 03:39:12.654992 29801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "6a45654d2bb445198a2f4897d9257b74"
format_stamp: "Formatted at 2026-08-09 03:39:12 on dist-test-slave-jq4t"
server_key: "1930ce82caf05ae8eb135c0236558b8d"
server_key_iv: "56705a922157b004640786c458dfea7f"
server_key_version: "encryptionkey@0"
I20260809 03:39:12.658946 29801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 03:39:12.661487 29828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:12.662729 29801 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:39:12.662863 29801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "6a45654d2bb445198a2f4897d9257b74"
format_stamp: "Formatted at 2026-08-09 03:39:12 on dist-test-slave-jq4t"
server_key: "1930ce82caf05ae8eb135c0236558b8d"
server_key_iv: "56705a922157b004640786c458dfea7f"
server_key_version: "encryptionkey@0"
I20260809 03:39:12.662971 29801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:12.692137 29801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:12.695190 29801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:12.695410 29801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:12.702992 29801 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:45437
I20260809 03:39:12.703030 29908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:45437 every 8 connection(s)
I20260809 03:39:12.704428 29801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:12.707840 29910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:12.708307 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 29801
I20260809 03:39:12.708462 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:12.708771 1921 external_mini_cluster.cc:1428] Setting key 331ae4a8e0da70c2c13976281c7fa1a7
I20260809 03:39:12.715852 29910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74: Bootstrap starting.
I20260809 03:39:12.718951 29910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74: Neither blocks nor log segments found. Creating new log.
Aug 09 03:39:12 dist-test-slave-jq4t krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:12.720024 29910 log.cc:824] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:12.722812 29910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74: No bootstrap required, opened a new log
I20260809 03:39:12.725364 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:12.710329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53772 (local address 127.1.224.126:45437)
0809 03:39:12.710788 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:12.710809 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:12.710872 (+ 63us) server_negotiation.cc:407] Connection header received
0809 03:39:12.711685 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:12.711712 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:12.712168 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:12.712593 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:12.713385 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:12.714974 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:12.715984 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:12.716727 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:12.716896 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:12.719942 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:12.719949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:12.719967 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:12.720002 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:12.722108 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:12.722680 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:12.722686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:12.722691 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:12.722811 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:12.723134 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:12.723136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:12.723137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:12.723594 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:12.723749 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:12.724107 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 03:39:12.724442 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":128,"threads_started":1}
I20260809 03:39:12.727125 29910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a45654d2bb445198a2f4897d9257b74" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45437 } }
I20260809 03:39:12.727367 29910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:12.727427 29910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a45654d2bb445198a2f4897d9257b74, State: Initialized, Role: FOLLOWER
I20260809 03:39:12.727954 29910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a45654d2bb445198a2f4897d9257b74" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45437 } }
I20260809 03:39:12.728124 29910 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:12.728196 29910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:12.728313 29910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:12.729393 29910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a45654d2bb445198a2f4897d9257b74" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45437 } }
I20260809 03:39:12.729765 29910 leader_election.cc:302] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a45654d2bb445198a2f4897d9257b74; no voters:
I20260809 03:39:12.730160 29910 leader_election.cc:288] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:12.730270 29916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:12.730513 29916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [term 1 LEADER]: Becoming Leader. State: Replica: 6a45654d2bb445198a2f4897d9257b74, State: Running, Role: LEADER
I20260809 03:39:12.730842 29916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a45654d2bb445198a2f4897d9257b74" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45437 } }
I20260809 03:39:12.731518 29910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:12.732613 29918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a45654d2bb445198a2f4897d9257b74. Latest consensus state: current_term: 1 leader_uuid: "6a45654d2bb445198a2f4897d9257b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a45654d2bb445198a2f4897d9257b74" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45437 } } }
I20260809 03:39:12.732749 29918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:12.732896 29917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a45654d2bb445198a2f4897d9257b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a45654d2bb445198a2f4897d9257b74" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 45437 } } }
I20260809 03:39:12.733067 29917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:12.733191 29931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:12.736107 29931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:12.741751 29931 catalog_manager.cc:1347] Generated new cluster ID: 464e14d87e464773907de70521906b31
I20260809 03:39:12.741827 29931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:12.756469 29931 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:12.757500 29931 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:12.766305 29931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74: Generated new TSK 0
I20260809 03:39:12.767235 29931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:39:12.807447 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45437
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:12.929428 29945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:12.929754 29945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:12.929833 29945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:12.934306 29945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:12.934412 29945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:12.934511 29945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:39:12.940284 29945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:12.941675 29945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:12.942858 29945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:12.953351 29956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:12.953403 29961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:12.953528 29945 server_base.cc:1034] running on GCE node
W20260809 03:39:12.955513 29957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:12.956005 29945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:12.956799 29945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:12.957983 29945 hybrid_clock.cc:648] HybridClock initialized: now 1786246752957957 us; error 40 us; skew 500 ppm
Aug 09 03:39:12 dist-test-slave-jq4t krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:12.961221 29945 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:39:12.962205 29945 webserver.cc:460] Webserver started at http://127.1.224.65:42771/ using document root <none> and password file <none>
I20260809 03:39:12.962831 29945 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:12.962899 29945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:12.963178 29945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:12.965489 29945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "852f3b31ddc748ccb4e42a64f86470d2"
format_stamp: "Formatted at 2026-08-09 03:39:12 on dist-test-slave-jq4t"
server_key: "b569a02cbfca343e219c844b2de73991"
server_key_iv: "dc1b825630b559fa5390027e89dce6bd"
server_key_version: "encryptionkey@0"
I20260809 03:39:12.966176 29945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "852f3b31ddc748ccb4e42a64f86470d2"
format_stamp: "Formatted at 2026-08-09 03:39:12 on dist-test-slave-jq4t"
server_key: "b569a02cbfca343e219c844b2de73991"
server_key_iv: "dc1b825630b559fa5390027e89dce6bd"
server_key_version: "encryptionkey@0"
I20260809 03:39:12.970552 29945 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 03:39:12.973299 29973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:12.974761 29945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:12.974933 29945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "852f3b31ddc748ccb4e42a64f86470d2"
format_stamp: "Formatted at 2026-08-09 03:39:12 on dist-test-slave-jq4t"
server_key: "b569a02cbfca343e219c844b2de73991"
server_key_iv: "dc1b825630b559fa5390027e89dce6bd"
server_key_version: "encryptionkey@0"
I20260809 03:39:12.975065 29945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:12.991067 29945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:12.995311 29945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:12.995568 29945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:12.996373 29945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:12.997408 29945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:12.997464 29945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:12.997541 29945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:12.997577 29945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:13.008140 29945 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:40973
I20260809 03:39:13.008164 30157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:40973 every 8 connection(s)
I20260809 03:39:13.009271 29945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:39:13.016841 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 29945
I20260809 03:39:13.016973 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:39:13.017314 1921 external_mini_cluster.cc:1428] Setting key 9f438a0695e01e140bb6ae6107cd13bb
Aug 09 03:39:13 dist-test-slave-jq4t krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:13.025148 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.011474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:40545 (local address 127.1.224.126:45437)
0809 03:39:13.011634 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:13.011638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:13.012687 (+ 1049us) server_negotiation.cc:407] Connection header received
0809 03:39:13.014139 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:13.014143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:13.014213 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:13.014322 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:13.015665 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.016225 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.017673 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.017928 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.018048 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.021107 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:13.021111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:13.021113 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:13.021140 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:13.022740 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:13.023332 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:13.023335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:13.023337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:13.023427 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:13.023813 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:13.023816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:13.023818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:13.024034 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:13.024152 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:13.024794 (+ 642us) server_negotiation.cc:299] Negotiation successful
0809 03:39:13.024952 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:39:13.026261 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.011929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:45437 (local address 127.1.224.65:40545)
0809 03:39:13.012458 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:13.012571 (+ 113us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:13.013865 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:13.014532 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:13.014541 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:13.014954 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:13.015485 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.015500 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.016374 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.016377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.017545 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.017557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.018105 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.018109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.018290 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.019070 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:13.019093 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:13.020904 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:13.022881 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:13.022889 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:13.022909 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:13.023210 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:13.023527 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:13.023531 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:13.023533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:13.023704 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:13.024153 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:13.024162 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:13.024463 (+ 301us) client_negotiation.cc:769] Sending connection context
0809 03:39:13.024735 (+ 272us) client_negotiation.cc:240] Negotiation successful
0809 03:39:13.025123 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":144,"threads_started":1}
I20260809 03:39:13.027837 30158 heartbeater.cc:347] Connected to a master server at 127.1.224.126:45437
I20260809 03:39:13.028141 30158 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:39:13.028791 30158 heartbeater.cc:511] Master 127.1.224.126:45437 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:39:13.030768 29852 ts_manager.cc:194] Registered new tserver with Master: 852f3b31ddc748ccb4e42a64f86470d2 (127.1.224.65:40973)
I20260809 03:39:13.032280 29852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:40545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:39:13.048349 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45437
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:13.168466 30164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:13.168797 30164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:13.168840 30164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:13.172887 30164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:13.172951 30164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:13.173008 30164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:39:13.178337 30164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:13.179430 30164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:13.180503 30164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:13.190395 30172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:13.190414 30171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:13.190403 30175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:13.190682 30164 server_base.cc:1034] running on GCE node
I20260809 03:39:13.191198 30164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:13.191802 30164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:13.193002 30164 hybrid_clock.cc:648] HybridClock initialized: now 1786246753192981 us; error 31 us; skew 500 ppm
Aug 09 03:39:13 dist-test-slave-jq4t krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246753, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:13.195762 30164 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:39:13.196779 30164 webserver.cc:460] Webserver started at http://127.1.224.66:33703/ using document root <none> and password file <none>
I20260809 03:39:13.197368 30164 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:13.197456 30164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:13.197693 30164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:13.199748 30164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "76e8a64ba94643c4a1294baad0e10f16"
format_stamp: "Formatted at 2026-08-09 03:39:13 on dist-test-slave-jq4t"
server_key: "3ee7f1367252161cd3ede55ab5049e8a"
server_key_iv: "e0563b91dbc05ee864e5f634fef71a46"
server_key_version: "encryptionkey@0"
I20260809 03:39:13.200331 30164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "76e8a64ba94643c4a1294baad0e10f16"
format_stamp: "Formatted at 2026-08-09 03:39:13 on dist-test-slave-jq4t"
server_key: "3ee7f1367252161cd3ede55ab5049e8a"
server_key_iv: "e0563b91dbc05ee864e5f634fef71a46"
server_key_version: "encryptionkey@0"
I20260809 03:39:13.204147 30164 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:39:13.206710 30188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:13.207955 30164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:39:13.208115 30164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "76e8a64ba94643c4a1294baad0e10f16"
format_stamp: "Formatted at 2026-08-09 03:39:13 on dist-test-slave-jq4t"
server_key: "3ee7f1367252161cd3ede55ab5049e8a"
server_key_iv: "e0563b91dbc05ee864e5f634fef71a46"
server_key_version: "encryptionkey@0"
I20260809 03:39:13.208235 30164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:13.225827 30164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:13.229780 30164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:13.230033 30164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:13.230738 30164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:13.231762 30164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:13.231853 30164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:13.231935 30164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:13.231962 30164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:13.241839 30164 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:38893
I20260809 03:39:13.241859 30374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:38893 every 8 connection(s)
I20260809 03:39:13.242837 30164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:39:13.247272 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 30164
I20260809 03:39:13.247419 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:39:13.247761 1921 external_mini_cluster.cc:1428] Setting key 14cddb1c58783c36f9c7cf709f2eb4a0
Aug 09 03:39:13 dist-test-slave-jq4t krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246753, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:13.259792 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.244814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:40165 (local address 127.1.224.126:45437)
0809 03:39:13.244987 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:13.244992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:13.245932 (+ 940us) server_negotiation.cc:407] Connection header received
0809 03:39:13.247296 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:13.247303 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:13.247377 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:13.247486 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:13.248969 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.249661 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.251526 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.251907 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.252029 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.255276 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:13.255283 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:13.255289 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:13.255328 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:13.257231 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:13.257828 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:13.257834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:13.257836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:13.257951 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:13.258352 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:13.258356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:13.258357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:13.258644 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:13.258769 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:13.259412 (+ 643us) server_negotiation.cc:299] Negotiation successful
0809 03:39:13.259577 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:39:13.260723 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.245278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:45437 (local address 127.1.224.66:40165)
0809 03:39:13.245766 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:13.245812 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:13.247062 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:13.247687 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:13.247694 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:13.248164 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:13.248815 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.248830 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.249805 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.249812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.251387 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.251404 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.252110 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.252116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.252358 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.253090 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:13.253118 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:13.255075 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:13.257386 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:13.257393 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:13.257408 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:13.257719 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:13.258079 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:13.258082 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:13.258083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:13.258242 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:13.258798 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:13.258805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:13.259124 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 03:39:13.259321 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 03:39:13.259629 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":131,"threads_started":1}
I20260809 03:39:13.262073 30375 heartbeater.cc:347] Connected to a master server at 127.1.224.126:45437
I20260809 03:39:13.262354 30375 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:13.262992 30375 heartbeater.cc:511] Master 127.1.224.126:45437 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
I20260809 03:39:13.264290 29852 ts_manager.cc:194] Registered new tserver with Master: 76e8a64ba94643c4a1294baad0e10f16 (127.1.224.66:38893)
I20260809 03:39:13.264983 29852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:40165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:39:13.279876 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45437
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:13.402750 30385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:13.403122 30385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:13.403213 30385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:13.407464 30385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:13.407584 30385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:13.407683 30385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:39:13.414103 30385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:45437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:13.415519 30385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:13.416781 30385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:13.427227 30393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:13.427224 30391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:13.427224 30395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:13.428668 30385 server_base.cc:1034] running on GCE node
I20260809 03:39:13.429149 30385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:13.429862 30385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:13.431061 30385 hybrid_clock.cc:648] HybridClock initialized: now 1786246753431021 us; error 50 us; skew 500 ppm
Aug 09 03:39:13 dist-test-slave-jq4t krb5kdc[29778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246753, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:13.434190 30385 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:39:13.435178 30385 webserver.cc:460] Webserver started at http://127.1.224.67:35311/ using document root <none> and password file <none>
I20260809 03:39:13.435768 30385 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:13.435837 30385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:13.436023 30385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:13.438364 30385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "44d425994e1747ec86ee52b7862c605f"
format_stamp: "Formatted at 2026-08-09 03:39:13 on dist-test-slave-jq4t"
server_key: "dbd8c893424aeebe582dd0cae3a5538e"
server_key_iv: "11c0e6de026d557dbfd0284a462169b2"
server_key_version: "encryptionkey@0"
I20260809 03:39:13.438930 30385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "44d425994e1747ec86ee52b7862c605f"
format_stamp: "Formatted at 2026-08-09 03:39:13 on dist-test-slave-jq4t"
server_key: "dbd8c893424aeebe582dd0cae3a5538e"
server_key_iv: "11c0e6de026d557dbfd0284a462169b2"
server_key_version: "encryptionkey@0"
I20260809 03:39:13.443120 30385 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 03:39:13.445803 30405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:13.447156 30385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 03:39:13.447291 30385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "44d425994e1747ec86ee52b7862c605f"
format_stamp: "Formatted at 2026-08-09 03:39:13 on dist-test-slave-jq4t"
server_key: "dbd8c893424aeebe582dd0cae3a5538e"
server_key_iv: "11c0e6de026d557dbfd0284a462169b2"
server_key_version: "encryptionkey@0"
I20260809 03:39:13.447410 30385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:13.464026 30385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:13.467496 30385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:13.467720 30385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:13.468441 30385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:13.469434 30385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:13.469485 30385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:13.469548 30385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:13.469583 30385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:13.478808 30385 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:32991
I20260809 03:39:13.478835 30579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:32991 every 8 connection(s)
I20260809 03:39:13.479794 30385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:39:13.488674 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 30385
I20260809 03:39:13.488837 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:39:13.489166 1921 external_mini_cluster.cc:1428] Setting key f1f2e2b96860c4947207fae0c98f79a4
Aug 09 03:39:13 dist-test-slave-jq4t krb5kdc[29778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246753, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:13.494588 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.481797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:60373 (local address 127.1.224.126:45437)
0809 03:39:13.481947 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:13.481951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:13.482824 (+ 873us) server_negotiation.cc:407] Connection header received
0809 03:39:13.484299 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:13.484303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:13.484368 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:13.484483 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:13.485807 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.486389 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.487629 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.487873 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.487996 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.491056 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:13.491060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:13.491063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:13.491096 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:13.492651 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:13.493171 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:13.493173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:13.493174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:13.493255 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:13.493557 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:13.493560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:13.493561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:13.493769 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:13.493845 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:13.494351 (+ 506us) server_negotiation.cc:299] Negotiation successful
0809 03:39:13.494461 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:39:13.495463 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.482188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:45437 (local address 127.1.224.67:60373)
0809 03:39:13.482658 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:13.482704 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:13.484048 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:13.484722 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:13.484729 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:13.485114 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:13.485643 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.485658 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.486531 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.486535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.487481 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.487492 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.488061 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.488065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.488248 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.489067 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:13.489085 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:13.490879 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:13.492760 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:13.492765 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:13.492780 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:13.493079 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:13.493333 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:13.493335 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:13.493336 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:13.493480 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:13.493856 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:13.493861 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:13.494135 (+ 274us) client_negotiation.cc:769] Sending connection context
0809 03:39:13.494302 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 03:39:13.494579 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":123,"threads_started":1}
I20260809 03:39:13.496688 30580 heartbeater.cc:347] Connected to a master server at 127.1.224.126:45437
I20260809 03:39:13.496948 30580 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:13.497470 30580 heartbeater.cc:511] Master 127.1.224.126:45437 requested a full tablet report, sending...
I20260809 03:39:13.498520 29852 ts_manager.cc:194] Registered new tserver with Master: 44d425994e1747ec86ee52b7862c605f (127.1.224.67:32991)
I20260809 03:39:13.499121 29852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:60373
I20260809 03:39:13.503309 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:39:13.512887 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.505278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53780 (local address 127.1.224.126:45437)
0809 03:39:13.505392 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:13.505396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:13.505561 (+ 165us) server_negotiation.cc:407] Connection header received
0809 03:39:13.505664 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:13.505666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:13.505704 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:13.505771 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:13.506467 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.506957 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.507728 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.507931 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.508014 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.509623 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:13.509626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:13.509628 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:13.509649 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:13.510941 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:13.511432 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:13.511435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:13.511437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:13.511541 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:13.511902 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:13.511905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:13.511907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:13.512124 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:13.512286 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:13.512614 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 03:39:13.512718 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:39:13.515849 29852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:39:13.519047 29852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:39:13.534343 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.528624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55670 (local address 127.1.224.65:40973)
0809 03:39:13.529006 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:13.529015 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:13.529055 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:39:13.529139 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:13.529144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:13.529365 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:13.530044 (+ 679us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:13.530188 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.531457 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.532897 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.533584 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.533742 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.533914 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:13.534087 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 03:39:13.534186 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":73,"threads_started":1}
I20260809 03:39:13.534333 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.527807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:40973 (local address 127.0.0.1:55670)
0809 03:39:13.528871 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:13.528886 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:13.528983 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:13.529519 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:13.529523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:13.529546 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:13.529880 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.529886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.531575 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.531579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.532753 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.532767 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.533741 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.533745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.533859 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.533898 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 03:39:13.534044 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 03:39:13.534177 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":995,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260809 03:39:13.535301 30600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.527750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:32991 (local address 127.0.0.1:53306)
0809 03:39:13.528250 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:13.528293 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:13.528405 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:13.529503 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:13.529510 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:13.529533 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:13.529743 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.529751 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.530979 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.530984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.532384 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.532395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.534877 (+ 2482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.534880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.534996 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.535021 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 03:39:13.535099 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 03:39:13.535168 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":19,"thread_start_us":98,"threads_started":1}
I20260809 03:39:13.535446 30602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.527896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53306 (local address 127.1.224.67:32991)
0809 03:39:13.528916 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:13.528931 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:13.528963 (+ 32us) server_negotiation.cc:407] Connection header received
0809 03:39:13.529052 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:13.529058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:13.529302 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:13.529724 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:13.529897 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.530845 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.533896 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.534742 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.534869 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.535027 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:13.535145 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 03:39:13.535256 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":129,"threads_started":1}
I20260809 03:39:13.535853 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.528557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:38893 (local address 127.0.0.1:44538)
0809 03:39:13.529174 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:13.529186 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:13.529261 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:13.530034 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:13.530037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:13.530058 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:13.530173 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.530181 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.532890 (+ 2709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.532892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.533737 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.533747 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.535044 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.535051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.535160 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.535185 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 03:39:13.535237 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 03:39:13.535304 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":93,"threads_started":1}
I20260809 03:39:13.536469 30608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.528798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44538 (local address 127.1.224.66:38893)
0809 03:39:13.529539 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:13.529551 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:13.529572 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:39:13.529647 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:13.529655 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:13.529849 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:13.530043 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:13.530287 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.531436 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.533858 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.534900 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.536001 (+ 1101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.536163 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:13.536221 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 03:39:13.536318 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":171,"threads_started":1}
I20260809 03:39:13.537863 30052 tablet_service.cc:1467] Processing CreateTablet for tablet 8675d42d1ed8480ca4fb4349cd8a7294 (DEFAULT_TABLE table=test-table [id=b42d4b0b6ef84e6bbaff3e4423cecf64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:13.538101 30475 tablet_service.cc:1467] Processing CreateTablet for tablet 8675d42d1ed8480ca4fb4349cd8a7294 (DEFAULT_TABLE table=test-table [id=b42d4b0b6ef84e6bbaff3e4423cecf64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:13.539050 30475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8675d42d1ed8480ca4fb4349cd8a7294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:13.539062 30052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8675d42d1ed8480ca4fb4349cd8a7294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:13.539151 30259 tablet_service.cc:1467] Processing CreateTablet for tablet 8675d42d1ed8480ca4fb4349cd8a7294 (DEFAULT_TABLE table=test-table [id=b42d4b0b6ef84e6bbaff3e4423cecf64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:13.540126 30259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8675d42d1ed8480ca4fb4349cd8a7294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:13.545392 30609 tablet_bootstrap.cc:492] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f: Bootstrap starting.
I20260809 03:39:13.547041 30612 tablet_bootstrap.cc:492] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16: Bootstrap starting.
I20260809 03:39:13.547798 30614 tablet_bootstrap.cc:492] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2: Bootstrap starting.
I20260809 03:39:13.547981 30609 tablet_bootstrap.cc:654] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:13.549486 30609 log.cc:824] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:13.550163 30614 tablet_bootstrap.cc:654] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:13.550773 30612 tablet_bootstrap.cc:654] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:13.551043 30614 log.cc:824] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:13.551671 30612 log.cc:824] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:13.552119 30609 tablet_bootstrap.cc:492] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f: No bootstrap required, opened a new log
I20260809 03:39:13.552327 30609 ts_tablet_manager.cc:1397] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260809 03:39:13.553220 30614 tablet_bootstrap.cc:492] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2: No bootstrap required, opened a new log
I20260809 03:39:13.553422 30614 ts_tablet_manager.cc:1397] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260809 03:39:13.553884 30612 tablet_bootstrap.cc:492] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16: No bootstrap required, opened a new log
I20260809 03:39:13.554132 30612 ts_tablet_manager.cc:1397] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 03:39:13.556752 30609 raft_consensus.cc:348] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d425994e1747ec86ee52b7862c605f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 32991 } } peers { permanent_uuid: "852f3b31ddc748ccb4e42a64f86470d2" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 40973 } } peers { permanent_uuid: "76e8a64ba94643c4a1294baad0e10f16" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 38893 } }
I20260809 03:39:13.557061 30609 raft_consensus.cc:374] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:13.557124 30609 raft_consensus.cc:729] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44d425994e1747ec86ee52b7862c605f, State: Initialized, Role: FOLLOWER
I20260809 03:39:13.557093 30612 raft_consensus.cc:348] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d425994e1747ec86ee52b7862c605f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 32991 } } peers { permanent_uuid: "852f3b31ddc748ccb4e42a64f86470d2" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 40973 } } peers { permanent_uuid: "76e8a64ba94643c4a1294baad0e10f16" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 38893 } }
I20260809 03:39:13.557364 30612 raft_consensus.cc:374] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:13.557433 30612 raft_consensus.cc:729] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76e8a64ba94643c4a1294baad0e10f16, State: Initialized, Role: FOLLOWER
I20260809 03:39:13.557659 30609 consensus_queue.cc:260] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d425994e1747ec86ee52b7862c605f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 32991 } } peers { permanent_uuid: "852f3b31ddc748ccb4e42a64f86470d2" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 40973 } } peers { permanent_uuid: "76e8a64ba94643c4a1294baad0e10f16" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 38893 } }
I20260809 03:39:13.557868 30612 consensus_queue.cc:260] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d425994e1747ec86ee52b7862c605f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 32991 } } peers { permanent_uuid: "852f3b31ddc748ccb4e42a64f86470d2" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 40973 } } peers { permanent_uuid: "76e8a64ba94643c4a1294baad0e10f16" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 38893 } }
I20260809 03:39:13.558040 30614 raft_consensus.cc:348] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d425994e1747ec86ee52b7862c605f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 32991 } } peers { permanent_uuid: "852f3b31ddc748ccb4e42a64f86470d2" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 40973 } } peers { permanent_uuid: "76e8a64ba94643c4a1294baad0e10f16" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 38893 } }
I20260809 03:39:13.558300 30614 raft_consensus.cc:374] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:13.558368 30614 raft_consensus.cc:729] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 852f3b31ddc748ccb4e42a64f86470d2, State: Initialized, Role: FOLLOWER
I20260809 03:39:13.558538 30580 heartbeater.cc:503] Master 127.1.224.126:45437 was elected leader, sending a full tablet report...
I20260809 03:39:13.558763 30609 ts_tablet_manager.cc:1428] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260809 03:39:13.558833 30612 ts_tablet_manager.cc:1428] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:39:13.558811 30614 consensus_queue.cc:260] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d425994e1747ec86ee52b7862c605f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 32991 } } peers { permanent_uuid: "852f3b31ddc748ccb4e42a64f86470d2" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 40973 } } peers { permanent_uuid: "76e8a64ba94643c4a1294baad0e10f16" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 38893 } }
I20260809 03:39:13.559187 30375 heartbeater.cc:503] Master 127.1.224.126:45437 was elected leader, sending a full tablet report...
I20260809 03:39:13.559710 30614 ts_tablet_manager.cc:1428] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260809 03:39:13.560066 30158 heartbeater.cc:503] Master 127.1.224.126:45437 was elected leader, sending a full tablet report...
I20260809 03:39:13.654083 30620 raft_consensus.cc:482] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:39:13.654333 30620 raft_consensus.cc:504] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d425994e1747ec86ee52b7862c605f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 32991 } } peers { permanent_uuid: "852f3b31ddc748ccb4e42a64f86470d2" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 40973 } } peers { permanent_uuid: "76e8a64ba94643c4a1294baad0e10f16" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 38893 } }
I20260809 03:39:13.655678 30620 leader_election.cc:288] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 852f3b31ddc748ccb4e42a64f86470d2 (127.1.224.65:40973), 76e8a64ba94643c4a1294baad0e10f16 (127.1.224.66:38893)
I20260809 03:39:13.659984 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.655954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:40973 (local address 127.1.224.67:52809)
0809 03:39:13.656116 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:13.656134 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:13.656249 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:13.656567 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:13.656571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:13.656593 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:13.656716 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.656722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.657696 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.657700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.658665 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.658676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.659524 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.659527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.659645 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.659668 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:39:13.659744 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 03:39:13.659817 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":21}
I20260809 03:39:13.660077 30608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.656172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:60409 (local address 127.1.224.66:38893)
0809 03:39:13.656335 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:13.656340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:13.656763 (+ 423us) server_negotiation.cc:407] Connection header received
0809 03:39:13.656868 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:13.656871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:13.656926 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:13.656986 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:13.657284 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.657857 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.658973 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.659487 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.659640 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.659696 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:13.659879 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 03:39:13.659949 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 03:39:13.660135 30624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.655961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:38893 (local address 127.1.224.67:60409)
0809 03:39:13.656671 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:13.656682 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:13.656769 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:13.657018 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:13.657021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:13.657039 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:13.657170 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.657176 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.657979 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.657981 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.658843 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:13.658854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.659636 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.659640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:13.659747 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.659768 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:39:13.659861 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 03:39:13.659934 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":292,"thread_start_us":174,"threads_started":1}
I20260809 03:39:13.660528 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.656088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:52809 (local address 127.1.224.65:40973)
0809 03:39:13.656243 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:13.656249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:13.656271 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:39:13.656365 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:13.656369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:13.656431 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:13.656538 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:13.656871 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.657581 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.658799 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.659375 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.659533 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.659596 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:13.660287 (+ 691us) server_negotiation.cc:299] Negotiation successful
0809 03:39:13.660377 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:39:13.660828 30293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8675d42d1ed8480ca4fb4349cd8a7294" candidate_uuid: "44d425994e1747ec86ee52b7862c605f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76e8a64ba94643c4a1294baad0e10f16" is_pre_election: true
I20260809 03:39:13.661152 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8675d42d1ed8480ca4fb4349cd8a7294" candidate_uuid: "44d425994e1747ec86ee52b7862c605f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "852f3b31ddc748ccb4e42a64f86470d2" is_pre_election: true
I20260809 03:39:13.661206 30293 raft_consensus.cc:2454] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44d425994e1747ec86ee52b7862c605f in term 0.
I20260809 03:39:13.661440 30091 raft_consensus.cc:2454] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44d425994e1747ec86ee52b7862c605f in term 0.
I20260809 03:39:13.661738 30408 leader_election.cc:302] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44d425994e1747ec86ee52b7862c605f, 76e8a64ba94643c4a1294baad0e10f16; no voters:
I20260809 03:39:13.662037 30620 raft_consensus.cc:2781] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:39:13.662140 30620 raft_consensus.cc:482] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:39:13.662187 30620 raft_consensus.cc:3037] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:13.663278 30620 raft_consensus.cc:504] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d425994e1747ec86ee52b7862c605f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 32991 } } peers { permanent_uuid: "852f3b31ddc748ccb4e42a64f86470d2" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 40973 } } peers { permanent_uuid: "76e8a64ba94643c4a1294baad0e10f16" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 38893 } }
I20260809 03:39:13.663637 30620 leader_election.cc:288] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [CANDIDATE]: Term 1 election: Requested vote from peers 852f3b31ddc748ccb4e42a64f86470d2 (127.1.224.65:40973), 76e8a64ba94643c4a1294baad0e10f16 (127.1.224.66:38893)
I20260809 03:39:13.663991 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8675d42d1ed8480ca4fb4349cd8a7294" candidate_uuid: "44d425994e1747ec86ee52b7862c605f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "852f3b31ddc748ccb4e42a64f86470d2"
I20260809 03:39:13.664024 30293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8675d42d1ed8480ca4fb4349cd8a7294" candidate_uuid: "44d425994e1747ec86ee52b7862c605f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76e8a64ba94643c4a1294baad0e10f16"
I20260809 03:39:13.664140 30293 raft_consensus.cc:3037] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:13.664134 30091 raft_consensus.cc:3037] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:13.665292 30091 raft_consensus.cc:2454] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44d425994e1747ec86ee52b7862c605f in term 1.
I20260809 03:39:13.665306 30293 raft_consensus.cc:2454] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44d425994e1747ec86ee52b7862c605f in term 1.
I20260809 03:39:13.665635 30409 leader_election.cc:302] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44d425994e1747ec86ee52b7862c605f, 852f3b31ddc748ccb4e42a64f86470d2; no voters:
I20260809 03:39:13.665810 30620 raft_consensus.cc:2781] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:13.666047 30620 raft_consensus.cc:686] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 1 LEADER]: Becoming Leader. State: Replica: 44d425994e1747ec86ee52b7862c605f, State: Running, Role: LEADER
I20260809 03:39:13.666464 30620 consensus_queue.cc:237] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d425994e1747ec86ee52b7862c605f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 32991 } } peers { permanent_uuid: "852f3b31ddc748ccb4e42a64f86470d2" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 40973 } } peers { permanent_uuid: "76e8a64ba94643c4a1294baad0e10f16" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 38893 } }
I20260809 03:39:13.669989 29852 catalog_manager.cc:5495] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f reported cstate change: term changed from 0 to 1, leader changed from <none> to 44d425994e1747ec86ee52b7862c605f (127.1.224.67). New cstate: current_term: 1 leader_uuid: "44d425994e1747ec86ee52b7862c605f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d425994e1747ec86ee52b7862c605f" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 32991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "852f3b31ddc748ccb4e42a64f86470d2" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 40973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76e8a64ba94643c4a1294baad0e10f16" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 38893 } health_report { overall_health: UNKNOWN } } }
I20260809 03:39:13.701881 30602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.696883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53318 (local address 127.1.224.67:32991)
0809 03:39:13.697018 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:13.697022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:13.697040 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:39:13.697157 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:13.697162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:13.697253 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:13.697399 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:13.697648 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.698498 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.699929 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:13.700248 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:13.700360 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:13.700756 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:13.701366 (+ 610us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:13.701534 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:13.701630 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 03:39:13.701676 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:39:13.709766 30293 raft_consensus.cc:1261] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16 [term 1 FOLLOWER]: Refusing update from remote peer 44d425994e1747ec86ee52b7862c605f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:13.710623 30620 consensus_queue.cc:1035] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [LEADER]: Connected to new peer: Peer: permanent_uuid: "76e8a64ba94643c4a1294baad0e10f16" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 38893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:13.710839 30091 raft_consensus.cc:1261] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2 [term 1 FOLLOWER]: Refusing update from remote peer 44d425994e1747ec86ee52b7862c605f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:13.711428 30620 consensus_queue.cc:1035] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [LEADER]: Connected to new peer: Peer: permanent_uuid: "852f3b31ddc748ccb4e42a64f86470d2" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 40973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:13.737317 29852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:39:13.737572 29852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:39:13.740639 29852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a45654d2bb445198a2f4897d9257b74: Sending DeleteTablet for 3 replicas of tablet 8675d42d1ed8480ca4fb4349cd8a7294
I20260809 03:39:13.741430 30475 tablet_service.cc:1514] Processing DeleteTablet for tablet 8675d42d1ed8480ca4fb4349cd8a7294 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:13 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:53306
I20260809 03:39:13.741645 30052 tablet_service.cc:1514] Processing DeleteTablet for tablet 8675d42d1ed8480ca4fb4349cd8a7294 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:13 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:55670
I20260809 03:39:13.742038 30646 tablet_replica.cc:330] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f: stopping tablet replica
I20260809 03:39:13.742223 30259 tablet_service.cc:1514] Processing DeleteTablet for tablet 8675d42d1ed8480ca4fb4349cd8a7294 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:13 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:44538
I20260809 03:39:13.742267 30647 tablet_replica.cc:330] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2: stopping tablet replica
I20260809 03:39:13.742409 30646 raft_consensus.cc:2229] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:39:13.742739 30647 raft_consensus.cc:2229] T 8675d42d1ed8480ca4fb4349cd8a7294 P 852f3b31ddc748ccb4e42a64f86470d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:39:13.742822 30646 raft_consensus.cc:2258] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:39:13.743263 30649 tablet_replica.cc:330] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16: stopping tablet replica
I20260809 03:39:13.743573 30649 raft_consensus.cc:2229] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:39:13.743901 30649 raft_consensus.cc:2258] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:39:13.743983 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 29945
I20260809 03:39:13.744585 30646 ts_tablet_manager.cc:1905] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:39:13.747771 30646 ts_tablet_manager.cc:1918] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:39:13.747867 30646 log.cc:1196] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/8675d42d1ed8480ca4fb4349cd8a7294
I20260809 03:39:13.748260 30646 ts_tablet_manager.cc:1939] T 8675d42d1ed8480ca4fb4349cd8a7294 P 44d425994e1747ec86ee52b7862c605f: Deleting consensus metadata
I20260809 03:39:13.748988 30649 ts_tablet_manager.cc:1905] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:39:13.750000 29830 catalog_manager.cc:4841] TS 44d425994e1747ec86ee52b7862c605f (127.1.224.67:32991): tablet 8675d42d1ed8480ca4fb4349cd8a7294 (table test-table [id=b42d4b0b6ef84e6bbaff3e4423cecf64]) successfully deleted
I20260809 03:39:13.752380 30649 ts_tablet_manager.cc:1918] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:39:13.752537 30649 log.cc:1196] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/8675d42d1ed8480ca4fb4349cd8a7294
I20260809 03:39:13.752915 30649 ts_tablet_manager.cc:1939] T 8675d42d1ed8480ca4fb4349cd8a7294 P 76e8a64ba94643c4a1294baad0e10f16: Deleting consensus metadata
I20260809 03:39:13.754554 29830 catalog_manager.cc:4841] TS 76e8a64ba94643c4a1294baad0e10f16 (127.1.224.66:38893): tablet 8675d42d1ed8480ca4fb4349cd8a7294 (table test-table [id=b42d4b0b6ef84e6bbaff3e4423cecf64]) successfully deleted
W20260809 03:39:13.765049 29831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:40973 (error 108)
I20260809 03:39:13.765120 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 30164
I20260809 03:39:13.777968 30601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:13.777491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:40973 (local address 127.0.0.1:55678)
0809 03:39:13.777706 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:13.777820 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:40973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260809 03:39:13.778366 29831 catalog_manager.cc:4584] TS 852f3b31ddc748ccb4e42a64f86470d2 (127.1.224.65:40973): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8675d42d1ed8480ca4fb4349cd8a7294: Network error: Client connection negotiation failed: client connection to 127.1.224.65:40973: connect: Connection refused (error 111)
I20260809 03:39:13.783267 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 30385
I20260809 03:39:13.798668 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 29801
2026-08-09T03:39:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2696 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T03:39:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:13Z Disabled control of system clock
I20260809 03:39:13.837457 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:43341
--webserver_interface=127.1.224.126
--webserver_port=0
--builtin_ntp_servers=127.1.224.84:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:43341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:39:13.957727 30655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:13.958042 30655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:13.958081 30655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:13.962185 30655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:13.962256 30655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:13.962272 30655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:13.962288 30655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:13.962304 30655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:13.967517 30655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:43341
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:43341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:13.968809 30655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:13.970332 30655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:13.979753 30665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:13.979758 30663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:13.980132 30655 server_base.cc:1034] running on GCE node
W20260809 03:39:13.980379 30668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:13.981045 30655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:13.982156 30655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:13.983312 30655 hybrid_clock.cc:648] HybridClock initialized: now 1786246753983294 us; error 31 us; skew 500 ppm
I20260809 03:39:13.990046 30655 webserver.cc:460] Webserver started at http://127.1.224.126:42169/ using document root <none> and password file <none>
I20260809 03:39:13.990819 30655 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:13.990909 30655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:13.991181 30655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:13.993542 30655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "755e284eea374cee85426134e3acdf19"
format_stamp: "Formatted at 2026-08-09 03:39:13 on dist-test-slave-jq4t"
server_key: "cbbedf021a8f8382a0784de93d2a5859"
server_key_iv: "93a22042c1705542f4b542a38094a3eb"
server_key_version: "encryptionkey@0"
I20260809 03:39:13.994170 30655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "755e284eea374cee85426134e3acdf19"
format_stamp: "Formatted at 2026-08-09 03:39:13 on dist-test-slave-jq4t"
server_key: "cbbedf021a8f8382a0784de93d2a5859"
server_key_iv: "93a22042c1705542f4b542a38094a3eb"
server_key_version: "encryptionkey@0"
I20260809 03:39:13.998111 30655 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:39:14.000617 30675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:14.001988 30655 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:39:14.002164 30655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "755e284eea374cee85426134e3acdf19"
format_stamp: "Formatted at 2026-08-09 03:39:13 on dist-test-slave-jq4t"
server_key: "cbbedf021a8f8382a0784de93d2a5859"
server_key_iv: "93a22042c1705542f4b542a38094a3eb"
server_key_version: "encryptionkey@0"
I20260809 03:39:14.002292 30655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:14.034322 30655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:14.035187 30655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:14.035413 30655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:14.043190 30655 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:43341
I20260809 03:39:14.043198 30750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:43341 every 8 connection(s)
I20260809 03:39:14.044242 30655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:14.044880 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 30655
I20260809 03:39:14.045054 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:14.045421 1921 external_mini_cluster.cc:1428] Setting key e194f52830a5a9a88a5267c317007273
I20260809 03:39:14.048233 30751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:14.054510 30751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19: Bootstrap starting.
I20260809 03:39:14.055418 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.047005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35628 (local address 127.1.224.126:43341)
0809 03:39:14.047702 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:14.047718 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:14.047757 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:39:14.048573 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:14.048593 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:14.049001 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:14.049357 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:14.049838 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.050895 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.051867 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.052373 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.052595 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.053559 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:14.053565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:14.053579 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:14.053995 (+ 416us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:14.054057 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:14.054078 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:14.054190 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:14.054319 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 03:39:14.054637 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":151,"threads_started":1}
I20260809 03:39:14.057271 30751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:14.058161 30751 log.cc:824] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:14.060315 30751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19: No bootstrap required, opened a new log
I20260809 03:39:14.064121 30751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755e284eea374cee85426134e3acdf19" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43341 } }
I20260809 03:39:14.064390 30751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:14.064469 30751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755e284eea374cee85426134e3acdf19, State: Initialized, Role: FOLLOWER
I20260809 03:39:14.064930 30751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755e284eea374cee85426134e3acdf19" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43341 } }
I20260809 03:39:14.065095 30751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:14.065173 30751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:14.065290 30751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:14.066256 30751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755e284eea374cee85426134e3acdf19" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43341 } }
I20260809 03:39:14.066617 30751 leader_election.cc:302] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 755e284eea374cee85426134e3acdf19; no voters:
I20260809 03:39:14.066974 30751 leader_election.cc:288] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:14.067216 30759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:14.067499 30759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [term 1 LEADER]: Becoming Leader. State: Replica: 755e284eea374cee85426134e3acdf19, State: Running, Role: LEADER
I20260809 03:39:14.067814 30759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755e284eea374cee85426134e3acdf19" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43341 } }
I20260809 03:39:14.068337 30751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:14.069667 30761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "755e284eea374cee85426134e3acdf19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755e284eea374cee85426134e3acdf19" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43341 } } }
I20260809 03:39:14.069695 30762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 755e284eea374cee85426134e3acdf19. Latest consensus state: current_term: 1 leader_uuid: "755e284eea374cee85426134e3acdf19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "755e284eea374cee85426134e3acdf19" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43341 } } }
I20260809 03:39:14.070463 30762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:14.070734 30761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:14.071141 30786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:14.074239 30786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:14.080905 30786 catalog_manager.cc:1347] Generated new cluster ID: 94e80748848c4d92aadbd0118c0316c4
I20260809 03:39:14.081015 30786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:14.089593 30786 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:14.090654 30786 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:14.101402 30786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19: Generated new TSK 0
I20260809 03:39:14.102274 30786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:14.105795 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43341
--builtin_ntp_servers=127.1.224.84:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:39:14.232097 30791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:14.232386 30791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:14.232427 30791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:14.236563 30791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:14.236658 30791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:14.236752 30791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:39:14.241609 30791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:14.242828 30791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:14.244364 30791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:14.254716 30800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:14.254870 30791 server_base.cc:1034] running on GCE node
W20260809 03:39:14.254717 30797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:14.254716 30805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:14.255601 30791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:14.256438 30791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:14.257696 30791 hybrid_clock.cc:648] HybridClock initialized: now 1786246754257665 us; error 42 us; skew 500 ppm
I20260809 03:39:14.259948 30791 webserver.cc:460] Webserver started at http://127.1.224.65:44327/ using document root <none> and password file <none>
I20260809 03:39:14.260685 30791 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:14.260785 30791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:14.261042 30791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:14.263389 30791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "d68e90d7f9b8430db456139e572dd9f9"
format_stamp: "Formatted at 2026-08-09 03:39:14 on dist-test-slave-jq4t"
server_key: "1f6bada44f29732e1b3383ae8b1c72d5"
server_key_iv: "6141be304d32094fafd5779a25763765"
server_key_version: "encryptionkey@0"
I20260809 03:39:14.264058 30791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "d68e90d7f9b8430db456139e572dd9f9"
format_stamp: "Formatted at 2026-08-09 03:39:14 on dist-test-slave-jq4t"
server_key: "1f6bada44f29732e1b3383ae8b1c72d5"
server_key_iv: "6141be304d32094fafd5779a25763765"
server_key_version: "encryptionkey@0"
I20260809 03:39:14.268393 30791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 03:39:14.271306 30812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:14.272675 30791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:14.272810 30791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "d68e90d7f9b8430db456139e572dd9f9"
format_stamp: "Formatted at 2026-08-09 03:39:14 on dist-test-slave-jq4t"
server_key: "1f6bada44f29732e1b3383ae8b1c72d5"
server_key_iv: "6141be304d32094fafd5779a25763765"
server_key_version: "encryptionkey@0"
I20260809 03:39:14.272941 30791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:14.290050 30791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:14.290864 30791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:14.291090 30791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:14.291762 30791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:14.292860 30791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:14.292930 30791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:14.292999 30791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:14.293040 30791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:14.302059 30791 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:39597
I20260809 03:39:14.302080 30979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:39597 every 8 connection(s)
I20260809 03:39:14.303057 30791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:39:14.304807 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 30791
I20260809 03:39:14.304916 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:39:14.305264 1921 external_mini_cluster.cc:1428] Setting key 3541878e650359043119a984a13658ff
I20260809 03:39:14.307576 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43341
--builtin_ntp_servers=127.1.224.84:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:39:14.314868 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.305083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:40659 (local address 127.1.224.126:43341)
0809 03:39:14.305250 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:14.305256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:14.306811 (+ 1555us) server_negotiation.cc:407] Connection header received
0809 03:39:14.308955 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:14.308958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:14.309009 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:14.309084 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:14.310196 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.311098 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.312333 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.312795 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.312919 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.314121 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:14.314127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:14.314131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:14.314151 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:14.314183 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:14.314197 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:14.314294 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:14.314527 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 03:39:14.314641 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 03:39:14.315640 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.305564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:43341 (local address 127.1.224.65:40659)
0809 03:39:14.306621 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:14.306683 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:14.308700 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:14.309297 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:14.309309 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:14.309428 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:14.310046 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.310060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.311247 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.311249 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.312193 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.312204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.312975 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.312978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.313128 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.313840 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:39:14.313842 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:14.313884 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:39:14.313952 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:14.314318 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:14.314328 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:14.314334 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:39:14.314462 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 03:39:14.314704 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":158,"threads_started":1}
I20260809 03:39:14.317028 30980 heartbeater.cc:347] Connected to a master server at 127.1.224.126:43341
I20260809 03:39:14.317277 30980 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:14.317859 30980 heartbeater.cc:511] Master 127.1.224.126:43341 requested a full tablet report, sending...
I20260809 03:39:14.319520 30696 ts_manager.cc:194] Registered new tserver with Master: d68e90d7f9b8430db456139e572dd9f9 (127.1.224.65:39597)
I20260809 03:39:14.320849 30696 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.224.65:40659
W20260809 03:39:14.429176 30984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:14.429473 30984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:14.429513 30984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:14.433674 30984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:14.433745 30984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:14.433802 30984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:39:14.438567 30984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:14.439708 30984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:14.441130 30984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:14.451216 30999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:14.451220 30996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:14.451261 31002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:14.451648 30984 server_base.cc:1034] running on GCE node
I20260809 03:39:14.452050 30984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:14.452742 30984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:14.453938 30984 hybrid_clock.cc:648] HybridClock initialized: now 1786246754453923 us; error 40 us; skew 500 ppm
I20260809 03:39:14.456112 30984 webserver.cc:460] Webserver started at http://127.1.224.66:33721/ using document root <none> and password file <none>
I20260809 03:39:14.456799 30984 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:14.456871 30984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:14.457057 30984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:14.459177 30984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "2f2cc2e1137d489ea0355da1c6a3d74c"
format_stamp: "Formatted at 2026-08-09 03:39:14 on dist-test-slave-jq4t"
server_key: "3e25e4de82e1a9cdb43e18cb52ab4609"
server_key_iv: "5f008e9ddb4dab91aebff77dbfcc2d52"
server_key_version: "encryptionkey@0"
I20260809 03:39:14.459720 30984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "2f2cc2e1137d489ea0355da1c6a3d74c"
format_stamp: "Formatted at 2026-08-09 03:39:14 on dist-test-slave-jq4t"
server_key: "3e25e4de82e1a9cdb43e18cb52ab4609"
server_key_iv: "5f008e9ddb4dab91aebff77dbfcc2d52"
server_key_version: "encryptionkey@0"
I20260809 03:39:14.463536 30984 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:39:14.466085 31009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:14.467383 30984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:39:14.467536 30984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "2f2cc2e1137d489ea0355da1c6a3d74c"
format_stamp: "Formatted at 2026-08-09 03:39:14 on dist-test-slave-jq4t"
server_key: "3e25e4de82e1a9cdb43e18cb52ab4609"
server_key_iv: "5f008e9ddb4dab91aebff77dbfcc2d52"
server_key_version: "encryptionkey@0"
I20260809 03:39:14.467658 30984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:14.478356 30984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:14.479051 30984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:14.479252 30984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:14.479857 30984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:14.480901 30984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:14.480970 30984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:14.481045 30984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:14.481086 30984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:14.490060 30984 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:37825
I20260809 03:39:14.490087 31175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:37825 every 8 connection(s)
I20260809 03:39:14.491078 30984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:39:14.496398 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 30984
I20260809 03:39:14.496564 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:39:14.496878 1921 external_mini_cluster.cc:1428] Setting key 140fcef4a8cb83e79e1432e178816c23
I20260809 03:39:14.499110 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43341
--builtin_ntp_servers=127.1.224.84:42279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:39:14.501363 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.492990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:60493 (local address 127.1.224.126:43341)
0809 03:39:14.493182 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:14.493187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:14.494272 (+ 1085us) server_negotiation.cc:407] Connection header received
0809 03:39:14.495783 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:14.495787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:14.495867 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:14.495983 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:14.497166 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.497803 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.499126 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.499512 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.499630 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.500837 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:14.500841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:14.500844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:14.500858 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:14.500886 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:14.500895 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:14.500972 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:14.501140 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 03:39:14.501212 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 03:39:14.502303 31179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.493442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:43341 (local address 127.1.224.66:60493)
0809 03:39:14.494125 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:14.494167 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:14.495542 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:14.496186 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:14.496212 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:14.496340 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:14.497011 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.497028 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.497949 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.497952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.498996 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.499007 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.499681 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.499684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.499833 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.500558 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:39:14.500560 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:14.500605 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:39:14.500676 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:14.500975 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:14.500988 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:14.500996 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 03:39:14.501139 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 03:39:14.501459 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":116,"threads_started":1}
I20260809 03:39:14.503537 31176 heartbeater.cc:347] Connected to a master server at 127.1.224.126:43341
I20260809 03:39:14.503837 31176 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:14.504657 31176 heartbeater.cc:511] Master 127.1.224.126:43341 requested a full tablet report, sending...
I20260809 03:39:14.506047 30696 ts_manager.cc:194] Registered new tserver with Master: 2f2cc2e1137d489ea0355da1c6a3d74c (127.1.224.66:37825)
I20260809 03:39:14.506979 30696 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.224.66:60493
W20260809 03:39:14.621548 31180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:14.621835 31180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:14.621876 31180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:14.625911 31180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:14.625986 31180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:14.626042 31180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:39:14.631062 31180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:42279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:43341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:14.632190 31180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:14.633594 31180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:14.643790 31190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:14.643913 31180 server_base.cc:1034] running on GCE node
W20260809 03:39:14.643792 31188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:14.643802 31192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:14.644563 31180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:14.645187 31180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:14.646345 31180 hybrid_clock.cc:648] HybridClock initialized: now 1786246754646313 us; error 49 us; skew 500 ppm
I20260809 03:39:14.648332 31180 webserver.cc:460] Webserver started at http://127.1.224.67:38575/ using document root <none> and password file <none>
I20260809 03:39:14.648988 31180 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:14.649056 31180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:14.649302 31180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:14.651489 31180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "e927a70eabd5453090bbf1e0b6180b29"
format_stamp: "Formatted at 2026-08-09 03:39:14 on dist-test-slave-jq4t"
server_key: "7315e55077f05fff187e676bb09cd526"
server_key_iv: "73a40dc85c133112eceef79d20d71cc9"
server_key_version: "encryptionkey@0"
I20260809 03:39:14.652078 31180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "e927a70eabd5453090bbf1e0b6180b29"
format_stamp: "Formatted at 2026-08-09 03:39:14 on dist-test-slave-jq4t"
server_key: "7315e55077f05fff187e676bb09cd526"
server_key_iv: "73a40dc85c133112eceef79d20d71cc9"
server_key_version: "encryptionkey@0"
I20260809 03:39:14.656056 31180 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 03:39:14.658699 31200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:14.659898 31180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 03:39:14.660044 31180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "e927a70eabd5453090bbf1e0b6180b29"
format_stamp: "Formatted at 2026-08-09 03:39:14 on dist-test-slave-jq4t"
server_key: "7315e55077f05fff187e676bb09cd526"
server_key_iv: "73a40dc85c133112eceef79d20d71cc9"
server_key_version: "encryptionkey@0"
I20260809 03:39:14.660158 31180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:14.678879 31180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:14.679697 31180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:14.679931 31180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:14.680616 31180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:14.681658 31180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:14.681730 31180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:14.681805 31180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:14.681845 31180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:14.691169 31180 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:45953
I20260809 03:39:14.691192 31367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:45953 every 8 connection(s)
I20260809 03:39:14.692155 31180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:39:14.697738 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 31180
I20260809 03:39:14.697892 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:39:14.698220 1921 external_mini_cluster.cc:1428] Setting key 593fcf7a5dda75d532544d419ab6ff0c
I20260809 03:39:14.701892 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.694125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:48257 (local address 127.1.224.126:43341)
0809 03:39:14.694281 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:14.694285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:14.695374 (+ 1089us) server_negotiation.cc:407] Connection header received
0809 03:39:14.696825 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:14.696828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:14.696878 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:14.696961 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:14.697910 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.698568 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.699811 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.700041 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.700154 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.701376 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:14.701380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:14.701387 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:14.701400 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:14.701420 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:14.701428 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:14.701512 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:14.701678 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 03:39:14.701746 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:39:14.702812 31370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.694545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:43341 (local address 127.1.224.67:48257)
0809 03:39:14.695220 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:14.695265 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:14.696598 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:14.697148 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:14.697155 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:14.697239 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:14.697738 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.697751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.698722 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.698726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.699666 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.699677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.700203 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.700207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.700429 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.701110 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:39:14.701112 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:14.701155 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:39:14.701221 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:14.701507 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:14.701518 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:14.701532 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 03:39:14.701676 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 03:39:14.701973 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":106,"threads_started":1}
I20260809 03:39:14.703917 31368 heartbeater.cc:347] Connected to a master server at 127.1.224.126:43341
I20260809 03:39:14.704169 31368 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:14.704736 31368 heartbeater.cc:511] Master 127.1.224.126:43341 requested a full tablet report, sending...
I20260809 03:39:14.705860 30696 ts_manager.cc:194] Registered new tserver with Master: e927a70eabd5453090bbf1e0b6180b29 (127.1.224.67:45953)
I20260809 03:39:14.706406 30696 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.224.67:48257
I20260809 03:39:14.712693 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:39:14.718802 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.714913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35638 (local address 127.1.224.126:43341)
0809 03:39:14.715044 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:14.715048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:14.715167 (+ 119us) server_negotiation.cc:407] Connection header received
0809 03:39:14.715278 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:14.715281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:14.715325 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:14.715403 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:14.715658 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.716182 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.717077 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.717294 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.717377 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.718271 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:14.718274 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:14.718276 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:14.718289 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:14.718310 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:14.718317 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:14.718440 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:14.718641 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 03:39:14.718677 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:39:14.722653 30696 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:39:14.726241 30696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:39:14.743191 31391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.737876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60964 (local address 127.1.224.66:37825)
0809 03:39:14.738265 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:14.738277 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:14.738300 (+ 23us) server_negotiation.cc:407] Connection header received
0809 03:39:14.738379 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:14.738383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:14.738554 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:14.738720 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:14.738903 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.740032 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.741867 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.742483 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.742617 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.742835 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:14.742899 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 03:39:14.743006 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":127,"threads_started":1}
I20260809 03:39:14.743187 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.737294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:37825 (local address 127.0.0.1:60964)
0809 03:39:14.737790 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:14.737804 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:14.737892 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:14.738689 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:14.738691 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:14.738711 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:14.738803 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.738810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.740590 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.740593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.741753 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.741760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.742615 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.742619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.742725 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.742764 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 03:39:14.742881 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 03:39:14.743027 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":1,"thread_start_us":115,"threads_started":1}
I20260809 03:39:14.743561 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.736820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49402 (local address 127.1.224.65:39597)
0809 03:39:14.738806 (+ 1986us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:14.738817 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:14.738842 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:39:14.738936 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:14.738941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:14.739076 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:14.739230 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:14.739413 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.740193 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.741697 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.742540 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.742666 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.742858 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:14.743259 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 03:39:14.743377 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1872,"thread_start_us":166,"threads_started":1}
I20260809 03:39:14.743777 31390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.737290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33198 (local address 127.1.224.67:45953)
0809 03:39:14.737736 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:14.737750 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:14.737782 (+ 32us) server_negotiation.cc:407] Connection header received
0809 03:39:14.737872 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:14.737878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:14.738124 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:14.738301 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:14.738728 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.740021 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.741741 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.742543 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.743009 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.743205 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:14.743257 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 03:39:14.743518 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":188,"threads_started":1}
I20260809 03:39:14.743870 31389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.736464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39597 (local address 127.0.0.1:49402)
0809 03:39:14.737580 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:14.737596 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:14.737699 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:14.739209 (+ 1510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:14.739212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:14.739235 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:14.739317 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.739323 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.740318 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.740320 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.741520 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.741531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.742928 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.742931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.743024 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.743043 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 03:39:14.743112 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 03:39:14.743181 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1045,"mutex_wait_us":49,"thread_start_us":106,"threads_started":1}
I20260809 03:39:14.744192 31385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.736464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:45953 (local address 127.0.0.1:33198)
0809 03:39:14.736917 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:14.736991 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:14.737148 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:14.738308 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:14.738314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:14.738340 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:14.738531 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.738540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.740146 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.740149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.741612 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.741624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.742696 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.742700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.742797 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.742820 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:39:14.742875 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 03:39:14.743912 (+ 1037us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":840,"thread_start_us":94,"threads_started":1}
I20260809 03:39:14.746562 31081 tablet_service.cc:1467] Processing CreateTablet for tablet ff315eb9d82d40e191022fe3f4995d9e (DEFAULT_TABLE table=test-table [id=9e454012ee6246d586f9f566e5767325]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:14.746647 30884 tablet_service.cc:1467] Processing CreateTablet for tablet ff315eb9d82d40e191022fe3f4995d9e (DEFAULT_TABLE table=test-table [id=9e454012ee6246d586f9f566e5767325]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:14.747064 31267 tablet_service.cc:1467] Processing CreateTablet for tablet ff315eb9d82d40e191022fe3f4995d9e (DEFAULT_TABLE table=test-table [id=9e454012ee6246d586f9f566e5767325]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:14.747684 31081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff315eb9d82d40e191022fe3f4995d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:14.747725 30884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff315eb9d82d40e191022fe3f4995d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:14.747973 31267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff315eb9d82d40e191022fe3f4995d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:14.754364 31393 tablet_bootstrap.cc:492] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c: Bootstrap starting.
I20260809 03:39:14.755968 31395 tablet_bootstrap.cc:492] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29: Bootstrap starting.
I20260809 03:39:14.756871 31394 tablet_bootstrap.cc:492] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9: Bootstrap starting.
I20260809 03:39:14.756927 31393 tablet_bootstrap.cc:654] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:14.758096 31393 log.cc:824] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:14.758379 31395 tablet_bootstrap.cc:654] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:14.759042 31394 tablet_bootstrap.cc:654] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:14.759271 31395 log.cc:824] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:14.759883 31394 log.cc:824] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:14.760571 31393 tablet_bootstrap.cc:492] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c: No bootstrap required, opened a new log
I20260809 03:39:14.760809 31393 ts_tablet_manager.cc:1397] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:39:14.761366 31395 tablet_bootstrap.cc:492] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29: No bootstrap required, opened a new log
I20260809 03:39:14.761584 31395 ts_tablet_manager.cc:1397] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:39:14.761999 31394 tablet_bootstrap.cc:492] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9: No bootstrap required, opened a new log
I20260809 03:39:14.762190 31394 ts_tablet_manager.cc:1397] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:39:14.765872 31393 raft_consensus.cc:348] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d68e90d7f9b8430db456139e572dd9f9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39597 } } peers { permanent_uuid: "e927a70eabd5453090bbf1e0b6180b29" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45953 } } peers { permanent_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37825 } }
I20260809 03:39:14.765857 31395 raft_consensus.cc:348] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d68e90d7f9b8430db456139e572dd9f9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39597 } } peers { permanent_uuid: "e927a70eabd5453090bbf1e0b6180b29" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45953 } } peers { permanent_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37825 } }
I20260809 03:39:14.766201 31393 raft_consensus.cc:374] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:14.766212 31395 raft_consensus.cc:374] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:14.766278 31395 raft_consensus.cc:729] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e927a70eabd5453090bbf1e0b6180b29, State: Initialized, Role: FOLLOWER
I20260809 03:39:14.766279 31393 raft_consensus.cc:729] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2cc2e1137d489ea0355da1c6a3d74c, State: Initialized, Role: FOLLOWER
I20260809 03:39:14.766479 31394 raft_consensus.cc:348] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d68e90d7f9b8430db456139e572dd9f9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39597 } } peers { permanent_uuid: "e927a70eabd5453090bbf1e0b6180b29" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45953 } } peers { permanent_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37825 } }
I20260809 03:39:14.766781 31394 raft_consensus.cc:374] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:14.766840 31394 raft_consensus.cc:729] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d68e90d7f9b8430db456139e572dd9f9, State: Initialized, Role: FOLLOWER
I20260809 03:39:14.766785 31393 consensus_queue.cc:260] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d68e90d7f9b8430db456139e572dd9f9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39597 } } peers { permanent_uuid: "e927a70eabd5453090bbf1e0b6180b29" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45953 } } peers { permanent_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37825 } }
I20260809 03:39:14.766785 31395 consensus_queue.cc:260] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d68e90d7f9b8430db456139e572dd9f9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39597 } } peers { permanent_uuid: "e927a70eabd5453090bbf1e0b6180b29" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45953 } } peers { permanent_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37825 } }
I20260809 03:39:14.767259 31394 consensus_queue.cc:260] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d68e90d7f9b8430db456139e572dd9f9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39597 } } peers { permanent_uuid: "e927a70eabd5453090bbf1e0b6180b29" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45953 } } peers { permanent_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37825 } }
I20260809 03:39:14.767813 31368 heartbeater.cc:503] Master 127.1.224.126:43341 was elected leader, sending a full tablet report...
I20260809 03:39:14.767861 31176 heartbeater.cc:503] Master 127.1.224.126:43341 was elected leader, sending a full tablet report...
I20260809 03:39:14.767948 31393 ts_tablet_manager.cc:1428] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:39:14.768826 31395 ts_tablet_manager.cc:1428] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 03:39:14.769044 31394 ts_tablet_manager.cc:1428] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 03:39:14.769407 30980 heartbeater.cc:503] Master 127.1.224.126:43341 was elected leader, sending a full tablet report...
W20260809 03:39:14.804435 30982 tablet.cc:2369] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:39:14.828907 31408 raft_consensus.cc:482] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:39:14.829218 31408 raft_consensus.cc:504] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d68e90d7f9b8430db456139e572dd9f9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39597 } } peers { permanent_uuid: "e927a70eabd5453090bbf1e0b6180b29" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45953 } } peers { permanent_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37825 } }
I20260809 03:39:14.830559 31408 leader_election.cc:288] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e927a70eabd5453090bbf1e0b6180b29 (127.1.224.67:45953), 2f2cc2e1137d489ea0355da1c6a3d74c (127.1.224.66:37825)
I20260809 03:39:14.835978 31390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.830973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:60447 (local address 127.1.224.67:45953)
0809 03:39:14.831148 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:14.831153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:14.831296 (+ 143us) server_negotiation.cc:407] Connection header received
0809 03:39:14.831477 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:14.831481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:14.831559 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:14.831652 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:14.831994 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.832822 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.834590 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.835199 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.835330 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.835385 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:14.835753 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 03:39:14.835827 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 03:39:14.835978 31391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.831464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:57977 (local address 127.1.224.66:37825)
0809 03:39:14.831652 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:14.831657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:14.831698 (+ 41us) server_negotiation.cc:407] Connection header received
0809 03:39:14.831843 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:14.831845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:14.831890 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:14.831976 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:14.832224 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.832945 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.834590 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.835201 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.835330 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.835385 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:14.835753 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 03:39:14.835827 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 03:39:14.836036 31409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.830868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:45953 (local address 127.1.224.65:60447)
0809 03:39:14.831153 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:14.831171 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:14.831340 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:14.831692 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:14.831695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:14.831725 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:14.831872 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.831884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.832926 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.832928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.834452 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.834464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.835350 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.835354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.835544 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.835626 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 03:39:14.835718 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 03:39:14.835842 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":179,"spinlock_wait_cycles":1792,"thread_start_us":89,"threads_started":1}
I20260809 03:39:14.836036 31410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.831303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:37825 (local address 127.1.224.65:57977)
0809 03:39:14.831609 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:14.831638 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:14.831736 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:14.831990 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:14.831994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:14.832013 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:14.832114 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.832118 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.833051 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.833054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.834456 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:14.834464 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.835350 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.835354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:14.835570 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.835634 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 03:39:14.835718 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 03:39:14.835836 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":223,"spinlock_wait_cycles":3584,"thread_start_us":97,"threads_started":1}
I20260809 03:39:14.836884 31292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff315eb9d82d40e191022fe3f4995d9e" candidate_uuid: "d68e90d7f9b8430db456139e572dd9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e927a70eabd5453090bbf1e0b6180b29" is_pre_election: true
I20260809 03:39:14.836884 31106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff315eb9d82d40e191022fe3f4995d9e" candidate_uuid: "d68e90d7f9b8430db456139e572dd9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c" is_pre_election: true
I20260809 03:39:14.837235 31292 raft_consensus.cc:2454] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d68e90d7f9b8430db456139e572dd9f9 in term 0.
I20260809 03:39:14.837235 31106 raft_consensus.cc:2454] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d68e90d7f9b8430db456139e572dd9f9 in term 0.
I20260809 03:39:14.837782 30814 leader_election.cc:302] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2cc2e1137d489ea0355da1c6a3d74c, d68e90d7f9b8430db456139e572dd9f9; no voters:
I20260809 03:39:14.838049 31408 raft_consensus.cc:2781] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:39:14.838153 31408 raft_consensus.cc:482] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:39:14.838218 31408 raft_consensus.cc:3037] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:14.839525 31408 raft_consensus.cc:504] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d68e90d7f9b8430db456139e572dd9f9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39597 } } peers { permanent_uuid: "e927a70eabd5453090bbf1e0b6180b29" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45953 } } peers { permanent_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37825 } }
I20260809 03:39:14.839896 31408 leader_election.cc:288] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [CANDIDATE]: Term 1 election: Requested vote from peers e927a70eabd5453090bbf1e0b6180b29 (127.1.224.67:45953), 2f2cc2e1137d489ea0355da1c6a3d74c (127.1.224.66:37825)
I20260809 03:39:14.840354 31106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff315eb9d82d40e191022fe3f4995d9e" candidate_uuid: "d68e90d7f9b8430db456139e572dd9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c"
I20260809 03:39:14.840354 31292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff315eb9d82d40e191022fe3f4995d9e" candidate_uuid: "d68e90d7f9b8430db456139e572dd9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e927a70eabd5453090bbf1e0b6180b29"
I20260809 03:39:14.840521 31292 raft_consensus.cc:3037] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:14.840520 31106 raft_consensus.cc:3037] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:14.841861 31292 raft_consensus.cc:2454] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d68e90d7f9b8430db456139e572dd9f9 in term 1.
I20260809 03:39:14.841861 31106 raft_consensus.cc:2454] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d68e90d7f9b8430db456139e572dd9f9 in term 1.
I20260809 03:39:14.842208 30814 leader_election.cc:302] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d68e90d7f9b8430db456139e572dd9f9, e927a70eabd5453090bbf1e0b6180b29; no voters:
I20260809 03:39:14.842375 31408 raft_consensus.cc:2781] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:14.842630 31408 raft_consensus.cc:686] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [term 1 LEADER]: Becoming Leader. State: Replica: d68e90d7f9b8430db456139e572dd9f9, State: Running, Role: LEADER
I20260809 03:39:14.843007 31408 consensus_queue.cc:237] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d68e90d7f9b8430db456139e572dd9f9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39597 } } peers { permanent_uuid: "e927a70eabd5453090bbf1e0b6180b29" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45953 } } peers { permanent_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37825 } }
I20260809 03:39:14.846516 30692 catalog_manager.cc:5495] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d68e90d7f9b8430db456139e572dd9f9 (127.1.224.65). New cstate: current_term: 1 leader_uuid: "d68e90d7f9b8430db456139e572dd9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d68e90d7f9b8430db456139e572dd9f9" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 39597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e927a70eabd5453090bbf1e0b6180b29" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37825 } health_report { overall_health: UNKNOWN } } }
I20260809 03:39:14.872915 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.868929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49404 (local address 127.1.224.65:39597)
0809 03:39:14.869066 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:14.869070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:14.869085 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:39:14.869274 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:14.869277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:14.869405 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:14.869537 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:14.869756 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.870282 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.871149 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:14.871382 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:14.871463 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:14.871888 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:14.872332 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:14.872450 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:14.872694 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 03:39:14.872749 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:39:14.879490 31292 raft_consensus.cc:1261] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29 [term 1 FOLLOWER]: Refusing update from remote peer d68e90d7f9b8430db456139e572dd9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:14.879490 31106 raft_consensus.cc:1261] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c [term 1 FOLLOWER]: Refusing update from remote peer d68e90d7f9b8430db456139e572dd9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:14.880261 31408 consensus_queue.cc:1035] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f2cc2e1137d489ea0355da1c6a3d74c" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 37825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:14.880465 31412 consensus_queue.cc:1035] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e927a70eabd5453090bbf1e0b6180b29" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:14.891808 31420 mvcc.cc:205] Tried to move back new op lower bound from 7316466707978526720 to 7316466707839737856. Current Snapshot: MvccSnapshot[applied={T|T < 7316466707978526720}]
I20260809 03:39:14.898262 30692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:39:14.898485 30692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:39:14.901209 30692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 755e284eea374cee85426134e3acdf19: Sending DeleteTablet for 3 replicas of tablet ff315eb9d82d40e191022fe3f4995d9e
I20260809 03:39:14.902029 30884 tablet_service.cc:1514] Processing DeleteTablet for tablet ff315eb9d82d40e191022fe3f4995d9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:14 UTC) from {username='slave'} at 127.0.0.1:49402
I20260809 03:39:14.902120 31267 tablet_service.cc:1514] Processing DeleteTablet for tablet ff315eb9d82d40e191022fe3f4995d9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:14 UTC) from {username='slave'} at 127.0.0.1:33198
I20260809 03:39:14.902163 31081 tablet_service.cc:1514] Processing DeleteTablet for tablet ff315eb9d82d40e191022fe3f4995d9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:14 UTC) from {username='slave'} at 127.0.0.1:60964
I20260809 03:39:14.902657 31435 tablet_replica.cc:330] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c: stopping tablet replica
I20260809 03:39:14.902786 31429 tablet_replica.cc:330] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9: stopping tablet replica
I20260809 03:39:14.903031 31435 raft_consensus.cc:2229] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:39:14.903198 31429 raft_consensus.cc:2229] T ff315eb9d82d40e191022fe3f4995d9e P d68e90d7f9b8430db456139e572dd9f9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:39:14.903383 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 30791
I20260809 03:39:14.903394 31435 raft_consensus.cc:2258] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:39:14.904245 31436 tablet_replica.cc:330] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29: stopping tablet replica
I20260809 03:39:14.906663 31436 raft_consensus.cc:2229] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:39:14.906991 31436 raft_consensus.cc:2258] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:39:14.909986 31436 ts_tablet_manager.cc:1905] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:39:14.911653 31435 ts_tablet_manager.cc:1905] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:39:14.913720 31436 ts_tablet_manager.cc:1918] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:39:14.913846 31436 log.cc:1196] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/ff315eb9d82d40e191022fe3f4995d9e
I20260809 03:39:14.913940 31435 ts_tablet_manager.cc:1918] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:39:14.914072 31435 log.cc:1196] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/ff315eb9d82d40e191022fe3f4995d9e
I20260809 03:39:14.914232 31436 ts_tablet_manager.cc:1939] T ff315eb9d82d40e191022fe3f4995d9e P e927a70eabd5453090bbf1e0b6180b29: Deleting consensus metadata
I20260809 03:39:14.914422 31435 ts_tablet_manager.cc:1939] T ff315eb9d82d40e191022fe3f4995d9e P 2f2cc2e1137d489ea0355da1c6a3d74c: Deleting consensus metadata
I20260809 03:39:14.915701 30677 catalog_manager.cc:4841] TS e927a70eabd5453090bbf1e0b6180b29 (127.1.224.67:45953): tablet ff315eb9d82d40e191022fe3f4995d9e (table test-table [id=9e454012ee6246d586f9f566e5767325]) successfully deleted
I20260809 03:39:14.915946 30677 catalog_manager.cc:4841] TS 2f2cc2e1137d489ea0355da1c6a3d74c (127.1.224.66:37825): tablet ff315eb9d82d40e191022fe3f4995d9e (table test-table [id=9e454012ee6246d586f9f566e5767325]) successfully deleted
W20260809 03:39:14.918135 30681 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:39597 (error 108)
I20260809 03:39:14.919094 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 30984
I20260809 03:39:14.919335 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:14.918964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:39597 (local address 127.0.0.1:49418)
0809 03:39:14.919135 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:14.919223 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:39597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 03:39:14.925909 30681 catalog_manager.cc:4584] TS d68e90d7f9b8430db456139e572dd9f9 (127.1.224.65:39597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff315eb9d82d40e191022fe3f4995d9e: Network error: Client connection negotiation failed: client connection to 127.1.224.65:39597: connect: Connection refused (error 111)
I20260809 03:39:14.938553 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 31180
I20260809 03:39:14.976332 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 30655
2026-08-09T03:39:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1174 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:15 dist-test-slave-jq4t krb5kdc[31445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:15 dist-test-slave-jq4t krb5kdc[31445](info): set up 2 sockets
Aug 09 03:39:15 dist-test-slave-jq4t krb5kdc[31445](info): commencing operation
krb5kdc: starting...
W20260809 03:39:16.233959 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:16 dist-test-slave-jq4t krb5kdc[31445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:39:16.371531 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:44523
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:44523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:16.491964 31472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:16.492333 31472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:16.492412 31472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:16.496881 31472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:16.497020 31472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:16.497058 31472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:16.497102 31472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:16.497138 31472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:16.503057 31472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:44523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:44523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:16.504299 31472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:16.505456 31472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:16.514324 31478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:16.514348 31477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:16.514294 31482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:16.514737 31472 server_base.cc:1034] running on GCE node
I20260809 03:39:16.515329 31472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:16.516448 31472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:16.517680 31472 hybrid_clock.cc:648] HybridClock initialized: now 1786246756517649 us; error 43 us; skew 500 ppm
Aug 09 03:39:16 dist-test-slave-jq4t krb5kdc[31445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246756, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:16.520740 31472 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:39:16.521720 31472 webserver.cc:460] Webserver started at http://127.1.224.126:43901/ using document root <none> and password file <none>
I20260809 03:39:16.522334 31472 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:16.522400 31472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:16.522627 31472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:16.524885 31472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "18daf6548696486ba4264eac5ce7be65"
format_stamp: "Formatted at 2026-08-09 03:39:16 on dist-test-slave-jq4t"
server_key: "f6660ba0ef7ed9383a0b0ad7e5e72e1b"
server_key_iv: "ce6150aa23ff9a5cac2491fa2f5ba80d"
server_key_version: "encryptionkey@0"
I20260809 03:39:16.525467 31472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "18daf6548696486ba4264eac5ce7be65"
format_stamp: "Formatted at 2026-08-09 03:39:16 on dist-test-slave-jq4t"
server_key: "f6660ba0ef7ed9383a0b0ad7e5e72e1b"
server_key_iv: "ce6150aa23ff9a5cac2491fa2f5ba80d"
server_key_version: "encryptionkey@0"
I20260809 03:39:16.529397 31472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:39:16.531919 31492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:16.533360 31472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:16.533514 31472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "18daf6548696486ba4264eac5ce7be65"
format_stamp: "Formatted at 2026-08-09 03:39:16 on dist-test-slave-jq4t"
server_key: "f6660ba0ef7ed9383a0b0ad7e5e72e1b"
server_key_iv: "ce6150aa23ff9a5cac2491fa2f5ba80d"
server_key_version: "encryptionkey@0"
I20260809 03:39:16.533627 31472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:16.566365 31472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:16.570062 31472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:16.570326 31472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:16.578361 31472 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:44523
I20260809 03:39:16.578382 31577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:44523 every 8 connection(s)
I20260809 03:39:16.579442 31472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:16.580127 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 31472
I20260809 03:39:16.580286 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:16.580649 1921 external_mini_cluster.cc:1428] Setting key dc4c218ac554f312102120fdcfcd0431
I20260809 03:39:16.582860 31578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:16.589946 31578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65: Bootstrap starting.
Aug 09 03:39:16 dist-test-slave-jq4t krb5kdc[31445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:16.592455 31578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:16.593294 31578 log.cc:824] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:16.595745 31578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65: No bootstrap required, opened a new log
I20260809 03:39:16.596283 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:16.582231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56386 (local address 127.1.224.126:44523)
0809 03:39:16.582661 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:16.582679 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:16.582717 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:39:16.583420 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:16.583437 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:16.583764 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:16.584041 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:16.585055 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:16.586075 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:16.587256 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:16.587736 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:16.587857 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:16.591005 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:16.591011 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:16.591026 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:16.591059 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:16.593020 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:16.593542 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:16.593547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:16.593553 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:16.593670 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:16.594011 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:16.594018 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:16.594022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:16.594524 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:16.594685 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:16.595071 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 03:39:16.595456 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":135,"threads_started":1}
I20260809 03:39:16.600031 31578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18daf6548696486ba4264eac5ce7be65" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44523 } }
I20260809 03:39:16.600307 31578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:16.600370 31578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18daf6548696486ba4264eac5ce7be65, State: Initialized, Role: FOLLOWER
I20260809 03:39:16.600926 31578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18daf6548696486ba4264eac5ce7be65" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44523 } }
I20260809 03:39:16.601112 31578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:16.601194 31578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:16.601315 31578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:16.602344 31578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18daf6548696486ba4264eac5ce7be65" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44523 } }
I20260809 03:39:16.602713 31578 leader_election.cc:302] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18daf6548696486ba4264eac5ce7be65; no voters:
I20260809 03:39:16.603106 31578 leader_election.cc:288] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:16.603289 31584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:16.603543 31584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [term 1 LEADER]: Becoming Leader. State: Replica: 18daf6548696486ba4264eac5ce7be65, State: Running, Role: LEADER
I20260809 03:39:16.603829 31584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18daf6548696486ba4264eac5ce7be65" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44523 } }
I20260809 03:39:16.604547 31578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:16.605772 31587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18daf6548696486ba4264eac5ce7be65. Latest consensus state: current_term: 1 leader_uuid: "18daf6548696486ba4264eac5ce7be65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18daf6548696486ba4264eac5ce7be65" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44523 } } }
I20260809 03:39:16.605777 31585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18daf6548696486ba4264eac5ce7be65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18daf6548696486ba4264eac5ce7be65" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44523 } } }
I20260809 03:39:16.606061 31585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:16.606099 31587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:16.606509 31598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:16.609479 31598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:16.615044 31598 catalog_manager.cc:1347] Generated new cluster ID: d1b8e03fe6104abeb04d1974047c99bb
I20260809 03:39:16.615118 31598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:16.626930 31598 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:16.628280 31598 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:16.637780 31598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65: Generated new TSK 0
I20260809 03:39:16.638767 31598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
Entry for principal kudu/127.1.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab.
I20260809 03:39:16.679585 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:44523
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:16.801870 31612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:16.802251 31612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:16.802337 31612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:16.806696 31612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:16.806779 31612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:16.806875 31612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:39:16.812369 31612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.65.keytab
--principal=kudu/127.1.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:44523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:16.813534 31612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:16.814648 31612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:16.825098 31624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:16.825085 31620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:16.825085 31622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:16.825129 31612 server_base.cc:1034] running on GCE node
I20260809 03:39:16.825817 31612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:16.826498 31612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:16.827685 31612 hybrid_clock.cc:648] HybridClock initialized: now 1786246756827655 us; error 64 us; skew 500 ppm
Aug 09 03:39:16 dist-test-slave-jq4t krb5kdc[31445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246756, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:16.830821 31612 init.cc:377] Logged in from keytab as kudu/127.1.224.65@KRBTEST.COM (short username kudu)
I20260809 03:39:16.831857 31612 webserver.cc:460] Webserver started at http://127.1.224.65:33259/ using document root <none> and password file <none>
I20260809 03:39:16.832468 31612 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:16.832594 31612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:16.832834 31612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:16.834968 31612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "c09120f3b2e7420bac607e5098631ea7"
format_stamp: "Formatted at 2026-08-09 03:39:16 on dist-test-slave-jq4t"
server_key: "9da87c803047293abef2b3dea9ee7eaf"
server_key_iv: "fbbe831320f962f688461b630a53ecee"
server_key_version: "encryptionkey@0"
I20260809 03:39:16.835546 31612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "c09120f3b2e7420bac607e5098631ea7"
format_stamp: "Formatted at 2026-08-09 03:39:16 on dist-test-slave-jq4t"
server_key: "9da87c803047293abef2b3dea9ee7eaf"
server_key_iv: "fbbe831320f962f688461b630a53ecee"
server_key_version: "encryptionkey@0"
I20260809 03:39:16.839598 31612 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260809 03:39:16.842152 31637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:16.843355 31612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:39:16.843480 31612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "c09120f3b2e7420bac607e5098631ea7"
format_stamp: "Formatted at 2026-08-09 03:39:16 on dist-test-slave-jq4t"
server_key: "9da87c803047293abef2b3dea9ee7eaf"
server_key_iv: "fbbe831320f962f688461b630a53ecee"
server_key_version: "encryptionkey@0"
I20260809 03:39:16.843585 31612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:16.855952 31612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:16.859032 31612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:16.859237 31612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:16.859885 31612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:16.860808 31612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:16.860893 31612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:16.860991 31612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:16.861038 31612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:16.870393 31612 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:34129
I20260809 03:39:16.870421 31796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:34129 every 8 connection(s)
I20260809 03:39:16.871348 31612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:39:16.877766 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 31612
I20260809 03:39:16.877929 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:39:16.878281 1921 external_mini_cluster.cc:1428] Setting key b78256aa1a6d031094d899f483c45485
Aug 09 03:39:16 dist-test-slave-jq4t krb5kdc[31445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246756, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.65@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:16.886655 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:16.873326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:41621 (local address 127.1.224.126:44523)
0809 03:39:16.873508 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:16.873513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:16.874389 (+ 876us) server_negotiation.cc:407] Connection header received
0809 03:39:16.875267 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:16.875270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:16.875321 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:16.875404 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:16.876681 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:16.877246 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:16.878961 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:16.879276 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:16.879394 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:16.882535 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:16.882542 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:16.882546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:16.882582 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:16.884127 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:16.884841 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:16.884844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:16.884845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:16.884923 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:16.885365 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:16.885367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:16.885368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:16.885583 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:16.885695 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:16.886408 (+ 713us) server_negotiation.cc:299] Negotiation successful
0809 03:39:16.886530 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 03:39:16.888074 31800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:16.873737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:44523 (local address 127.1.224.65:41621)
0809 03:39:16.874224 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:16.874268 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:16.875065 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:16.875589 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:16.875596 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:16.875985 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:16.876531 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:16.876548 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:16.877381 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:16.877384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:16.878818 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:16.878828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:16.879434 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:16.879437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:16.879621 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:16.880451 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:16.880513 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:16.882351 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:16.884257 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:16.884267 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:16.884285 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:16.884738 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:16.885021 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:16.885027 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:16.885032 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:16.885256 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:16.885727 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:16.885737 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:16.886121 (+ 384us) client_negotiation.cc:769] Sending connection context
0809 03:39:16.886359 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 03:39:16.886786 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":118,"threads_started":1}
I20260809 03:39:16.889531 31798 heartbeater.cc:347] Connected to a master server at 127.1.224.126:44523
I20260809 03:39:16.889801 31798 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.224.66@KRBTEST.COM; defaulting to no policy
I20260809 03:39:16.890347 31798 heartbeater.cc:511] Master 127.1.224.126:44523 requested a full tablet report, sending...
I20260809 03:39:16.892058 31520 ts_manager.cc:194] Registered new tserver with Master: c09120f3b2e7420bac607e5098631ea7 (127.1.224.65:34129)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.66@KRBTEST.COM" created.
I20260809 03:39:16.893462 31520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.65@KRBTEST.COM'} at 127.1.224.65:41621
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
Entry for principal kudu/127.1.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab.
I20260809 03:39:16.912285 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:44523
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:17.032265 31805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:17.032606 31805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:17.032658 31805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:17.036789 31805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:17.036887 31805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:17.036980 31805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:39:17.042699 31805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.66.keytab
--principal=kudu/127.1.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:44523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:17.043907 31805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:17.045089 31805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:17.055881 31810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:17.055907 31811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:17.055905 31813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:17.056157 31805 server_base.cc:1034] running on GCE node
I20260809 03:39:17.056707 31805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:17.057408 31805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:17.058591 31805 hybrid_clock.cc:648] HybridClock initialized: now 1786246757058568 us; error 33 us; skew 500 ppm
Aug 09 03:39:17 dist-test-slave-jq4t krb5kdc[31445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246757, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:17.061919 31805 init.cc:377] Logged in from keytab as kudu/127.1.224.66@KRBTEST.COM (short username kudu)
I20260809 03:39:17.062873 31805 webserver.cc:460] Webserver started at http://127.1.224.66:33651/ using document root <none> and password file <none>
I20260809 03:39:17.063550 31805 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:17.063620 31805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:17.063804 31805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:17.066017 31805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "ffc09e5c60f241c0943744cce3522b65"
format_stamp: "Formatted at 2026-08-09 03:39:17 on dist-test-slave-jq4t"
server_key: "c37919c501b217a685a68003f85f97f3"
server_key_iv: "371b71dfb0084fdea7ecd1b049109ec6"
server_key_version: "encryptionkey@0"
I20260809 03:39:17.066561 31805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "ffc09e5c60f241c0943744cce3522b65"
format_stamp: "Formatted at 2026-08-09 03:39:17 on dist-test-slave-jq4t"
server_key: "c37919c501b217a685a68003f85f97f3"
server_key_iv: "371b71dfb0084fdea7ecd1b049109ec6"
server_key_version: "encryptionkey@0"
I20260809 03:39:17.070500 31805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:39:17.073190 31822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:17.074527 31805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:17.074640 31805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "ffc09e5c60f241c0943744cce3522b65"
format_stamp: "Formatted at 2026-08-09 03:39:17 on dist-test-slave-jq4t"
server_key: "c37919c501b217a685a68003f85f97f3"
server_key_iv: "371b71dfb0084fdea7ecd1b049109ec6"
server_key_version: "encryptionkey@0"
I20260809 03:39:17.074740 31805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:17.099932 31805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:17.103304 31805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:17.103477 31805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:17.104071 31805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:17.105049 31805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:17.105103 31805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:17.105199 31805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:17.105232 31805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:17.114858 31805 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:33837
I20260809 03:39:17.114879 31997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:33837 every 8 connection(s)
I20260809 03:39:17.115818 31805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:39:17.120774 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 31805
I20260809 03:39:17.120889 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:39:17.121227 1921 external_mini_cluster.cc:1428] Setting key e95333ef2b983d8caf8caa29d275bdd9
Aug 09 03:39:17 dist-test-slave-jq4t krb5kdc[31445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246757, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.66@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:17.131513 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.117767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:45691 (local address 127.1.224.126:44523)
0809 03:39:17.117958 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:17.117963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:17.118816 (+ 853us) server_negotiation.cc:407] Connection header received
0809 03:39:17.119707 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:17.119710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:17.119761 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:17.119839 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:17.121246 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.121961 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.123502 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.123900 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.124031 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.126966 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:17.126973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:17.126980 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:17.127018 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:17.129012 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:17.129579 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:17.129585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:17.129589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:17.129705 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:17.130102 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:17.130109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:17.130114 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:17.130435 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:17.130563 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:17.131151 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 03:39:17.131309 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 03:39:17.132334 32001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.118172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:44523 (local address 127.1.224.66:45691)
0809 03:39:17.118651 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:17.118696 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:17.119510 (+ 814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:17.120023 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:17.120029 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:17.120401 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:17.121088 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.121107 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.122148 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.122151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.123365 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.123377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.124068 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.124071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.124239 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.124956 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:17.124986 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:17.126779 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:17.129156 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:17.129162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:17.129176 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:17.129457 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:17.129818 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:17.129820 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:17.129822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:17.130002 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:17.130570 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:17.130576 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:17.130865 (+ 289us) client_negotiation.cc:769] Sending connection context
0809 03:39:17.131052 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 03:39:17.131347 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":126,"threads_started":1}
WARNING: no policy specified for kudu/127.1.224.67@KRBTEST.COM; defaulting to no policy
I20260809 03:39:17.133670 31999 heartbeater.cc:347] Connected to a master server at 127.1.224.126:44523
I20260809 03:39:17.133960 31999 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:17.134557 31999 heartbeater.cc:511] Master 127.1.224.126:44523 requested a full tablet report, sending...
I20260809 03:39:17.135699 31520 ts_manager.cc:194] Registered new tserver with Master: ffc09e5c60f241c0943744cce3522b65 (127.1.224.66:33837)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.67@KRBTEST.COM" created.
I20260809 03:39:17.136386 31520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.66@KRBTEST.COM'} at 127.1.224.66:45691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
Entry for principal kudu/127.1.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab.
I20260809 03:39:17.155848 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:44523
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:17.276592 32006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:17.276912 32006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:17.276959 32006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:17.281260 32006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:17.281347 32006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:17.281410 32006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:39:17.286999 32006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.67.keytab
--principal=kudu/127.1.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:44523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:17.288193 32006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:17.289409 32006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:17.300323 32020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:17.300387 32006 server_base.cc:1034] running on GCE node
W20260809 03:39:17.300309 32016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:17.300302 32018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:17.301098 32006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:17.301743 32006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:17.302902 32006 hybrid_clock.cc:648] HybridClock initialized: now 1786246757302887 us; error 27 us; skew 500 ppm
Aug 09 03:39:17 dist-test-slave-jq4t krb5kdc[31445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246757, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:17.305998 32006 init.cc:377] Logged in from keytab as kudu/127.1.224.67@KRBTEST.COM (short username kudu)
I20260809 03:39:17.307009 32006 webserver.cc:460] Webserver started at http://127.1.224.67:44565/ using document root <none> and password file <none>
I20260809 03:39:17.307627 32006 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:17.307720 32006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:17.307960 32006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:17.310189 32006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "1a8acf9b759741e3916573c40fa373c0"
format_stamp: "Formatted at 2026-08-09 03:39:17 on dist-test-slave-jq4t"
server_key: "18ee230993d992ce463a9e26efb0f9aa"
server_key_iv: "1ec0c2e5b2352ffaf8f3a69a9124bf1d"
server_key_version: "encryptionkey@0"
I20260809 03:39:17.310787 32006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "1a8acf9b759741e3916573c40fa373c0"
format_stamp: "Formatted at 2026-08-09 03:39:17 on dist-test-slave-jq4t"
server_key: "18ee230993d992ce463a9e26efb0f9aa"
server_key_iv: "1ec0c2e5b2352ffaf8f3a69a9124bf1d"
server_key_version: "encryptionkey@0"
I20260809 03:39:17.314657 32006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:39:17.317240 32032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:17.318516 32006 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:39:17.318668 32006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "1a8acf9b759741e3916573c40fa373c0"
format_stamp: "Formatted at 2026-08-09 03:39:17 on dist-test-slave-jq4t"
server_key: "18ee230993d992ce463a9e26efb0f9aa"
server_key_iv: "1ec0c2e5b2352ffaf8f3a69a9124bf1d"
server_key_version: "encryptionkey@0"
I20260809 03:39:17.318786 32006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:17.345032 32006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:17.348938 32006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:17.349177 32006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:17.349881 32006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:17.350862 32006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:17.350935 32006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:17.351032 32006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:17.351073 32006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:17.360574 32006 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:45847
I20260809 03:39:17.360620 32222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:45847 every 8 connection(s)
I20260809 03:39:17.361567 32006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:39:17.364389 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 32006
I20260809 03:39:17.364573 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:39:17.364933 1921 external_mini_cluster.cc:1428] Setting key 32c40923b9f3b8e46c10b40cc59ad380
Aug 09 03:39:17 dist-test-slave-jq4t krb5kdc[31445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246757, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.67@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:17.376001 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.363612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:37675 (local address 127.1.224.126:44523)
0809 03:39:17.363753 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:17.363757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:17.364821 (+ 1064us) server_negotiation.cc:407] Connection header received
0809 03:39:17.366006 (+ 1185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:17.366010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:17.366072 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:17.366152 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:17.367615 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.368157 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.369450 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.369684 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.369791 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.372596 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:17.372599 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:17.372602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:17.372633 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:17.374122 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:17.374606 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:17.374609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:17.374610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:17.374689 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:17.374986 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:17.374988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:17.374989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:17.375180 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:17.375256 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:17.375759 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 03:39:17.375868 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:39:17.377000 32225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.363967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:44523 (local address 127.1.224.67:37675)
0809 03:39:17.364611 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:17.364683 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:17.365764 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:17.366341 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:17.366349 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:17.366857 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:17.367471 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.367486 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.368287 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.368290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.369309 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.369320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.369855 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.369860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.370047 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.370681 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:17.370700 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:17.372337 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:17.374227 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:17.374232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:17.374246 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:17.374516 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:17.374767 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:17.374769 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:17.374771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:17.374908 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:17.375267 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:17.375272 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:17.375545 (+ 273us) client_negotiation.cc:769] Sending connection context
0809 03:39:17.375711 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 03:39:17.375991 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":178,"threads_started":1}
I20260809 03:39:17.378410 32223 heartbeater.cc:347] Connected to a master server at 127.1.224.126:44523
I20260809 03:39:17.378700 32223 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:17.379407 32223 heartbeater.cc:511] Master 127.1.224.126:44523 requested a full tablet report, sending...
I20260809 03:39:17.380554 31520 ts_manager.cc:194] Registered new tserver with Master: 1a8acf9b759741e3916573c40fa373c0 (127.1.224.67:45847)
I20260809 03:39:17.381158 31520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.224.67@KRBTEST.COM'} at 127.1.224.67:37675
I20260809 03:39:17.391317 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:39:17.401358 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.393496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56400 (local address 127.1.224.126:44523)
0809 03:39:17.393706 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:17.393711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:17.393884 (+ 173us) server_negotiation.cc:407] Connection header received
0809 03:39:17.394058 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:17.394061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:17.394113 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:17.394197 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:17.394965 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.395532 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.396392 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.396680 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.396788 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.398225 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:17.398229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:17.398232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:17.398268 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:17.399494 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:17.399957 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:17.399961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:17.399963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:17.400051 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:17.400327 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:17.400331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:17.400332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:17.400662 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:17.400810 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:17.401061 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 03:39:17.401188 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 03:39:17.404707 31520 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:39:17.408133 31520 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:39:17.424453 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.418271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:33837 (local address 127.0.0.1:37016)
0809 03:39:17.419642 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:17.419657 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:17.419764 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:17.420287 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:17.420290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:17.420307 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:17.420420 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.420428 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.421537 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.421542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.422908 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.422919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.423949 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.423952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.424033 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.424065 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 03:39:17.424158 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 03:39:17.424247 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303,"mutex_wait_us":1,"thread_start_us":128,"threads_started":1}
I20260809 03:39:17.424504 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.418823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37016 (local address 127.1.224.66:33837)
0809 03:39:17.419228 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:17.419253 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:17.419874 (+ 621us) server_negotiation.cc:407] Connection header received
0809 03:39:17.419959 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:17.419967 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:17.420147 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:17.420294 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:17.420614 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.421395 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.423044 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.423836 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.423948 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.424107 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:17.424208 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 03:39:17.424325 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":105,"threads_started":1}
I20260809 03:39:17.424933 32248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.419243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52792 (local address 127.1.224.67:45847)
0809 03:39:17.419579 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:17.419590 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:17.419801 (+ 211us) server_negotiation.cc:407] Connection header received
0809 03:39:17.420095 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:17.420104 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:17.420293 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:17.420460 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:17.420750 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.421858 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.423393 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.424229 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.424332 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.424461 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:17.424662 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 03:39:17.424772 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":147,"threads_started":1}
I20260809 03:39:17.425333 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.419067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:45847 (local address 127.0.0.1:52792)
0809 03:39:17.419710 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:17.419724 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:17.419941 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:17.420482 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:17.420523 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:17.420544 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:17.420633 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.420639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.421995 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.421998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.423263 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.423271 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.424354 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.424357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.424437 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.424462 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 03:39:17.425146 (+ 684us) client_negotiation.cc:240] Negotiation successful
0809 03:39:17.425228 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":36,"spinlock_wait_cycles":896,"thread_start_us":101,"threads_started":1}
I20260809 03:39:17.426311 32240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.418103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:34129 (local address 127.0.0.1:38588)
0809 03:39:17.418646 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:17.418699 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:17.418831 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:17.419841 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:17.419858 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:17.419894 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:17.420064 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.420072 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.421871 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.421877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.423259 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.423271 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.425879 (+ 2608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.425883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.425982 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.426040 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 03:39:17.426097 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 03:39:17.426174 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":78,"spinlock_wait_cycles":384,"thread_start_us":110,"threads_started":1}
I20260809 03:39:17.426920 32244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.418273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38588 (local address 127.1.224.65:34129)
0809 03:39:17.419303 (+ 1030us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:17.419309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:17.419333 (+ 24us) server_negotiation.cc:407] Connection header received
0809 03:39:17.419432 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:17.419438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:17.419660 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:17.420232 (+ 572us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:17.420388 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.421729 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.424927 (+ 3198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.425743 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.426597 (+ 854us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.426707 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:17.426746 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 03:39:17.426812 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":934,"thread_start_us":137,"threads_started":1}
I20260809 03:39:17.428150 32115 tablet_service.cc:1467] Processing CreateTablet for tablet eb53dc0f708c4189b4daba0ac3de56a3 (DEFAULT_TABLE table=test-table [id=c2e24af21c544363a515454f5be2e8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:17.427945 31908 tablet_service.cc:1467] Processing CreateTablet for tablet eb53dc0f708c4189b4daba0ac3de56a3 (DEFAULT_TABLE table=test-table [id=c2e24af21c544363a515454f5be2e8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:17.429229 32115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb53dc0f708c4189b4daba0ac3de56a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:17.429224 31908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb53dc0f708c4189b4daba0ac3de56a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:17.429090 31708 tablet_service.cc:1467] Processing CreateTablet for tablet eb53dc0f708c4189b4daba0ac3de56a3 (DEFAULT_TABLE table=test-table [id=c2e24af21c544363a515454f5be2e8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:17.430158 31708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb53dc0f708c4189b4daba0ac3de56a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:17.436165 32249 tablet_bootstrap.cc:492] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7: Bootstrap starting.
I20260809 03:39:17.437091 32257 tablet_bootstrap.cc:492] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0: Bootstrap starting.
I20260809 03:39:17.439635 32249 tablet_bootstrap.cc:654] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:17.440083 32257 tablet_bootstrap.cc:654] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:17.440980 32257 log.cc:824] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:17.440981 32249 log.cc:824] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:17.441769 32251 tablet_bootstrap.cc:492] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65: Bootstrap starting.
I20260809 03:39:17.443125 32257 tablet_bootstrap.cc:492] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0: No bootstrap required, opened a new log
I20260809 03:39:17.443346 32257 ts_tablet_manager.cc:1397] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:39:17.443555 32249 tablet_bootstrap.cc:492] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7: No bootstrap required, opened a new log
I20260809 03:39:17.443746 32249 ts_tablet_manager.cc:1397] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260809 03:39:17.444643 32251 tablet_bootstrap.cc:654] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:17.445551 32251 log.cc:824] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:17.446734 32257 raft_consensus.cc:348] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09120f3b2e7420bac607e5098631ea7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34129 } } peers { permanent_uuid: "ffc09e5c60f241c0943744cce3522b65" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33837 } } peers { permanent_uuid: "1a8acf9b759741e3916573c40fa373c0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45847 } }
I20260809 03:39:17.447306 32257 raft_consensus.cc:374] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:17.447367 32257 raft_consensus.cc:729] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a8acf9b759741e3916573c40fa373c0, State: Initialized, Role: FOLLOWER
I20260809 03:39:17.447957 32251 tablet_bootstrap.cc:492] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65: No bootstrap required, opened a new log
I20260809 03:39:17.447851 32257 consensus_queue.cc:260] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09120f3b2e7420bac607e5098631ea7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34129 } } peers { permanent_uuid: "ffc09e5c60f241c0943744cce3522b65" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33837 } } peers { permanent_uuid: "1a8acf9b759741e3916573c40fa373c0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45847 } }
I20260809 03:39:17.448154 32251 ts_tablet_manager.cc:1397] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:39:17.448405 32249 raft_consensus.cc:348] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09120f3b2e7420bac607e5098631ea7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34129 } } peers { permanent_uuid: "ffc09e5c60f241c0943744cce3522b65" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33837 } } peers { permanent_uuid: "1a8acf9b759741e3916573c40fa373c0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45847 } }
I20260809 03:39:17.448704 32249 raft_consensus.cc:374] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:17.448764 32249 raft_consensus.cc:729] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c09120f3b2e7420bac607e5098631ea7, State: Initialized, Role: FOLLOWER
I20260809 03:39:17.449021 32257 ts_tablet_manager.cc:1428] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260809 03:39:17.449218 32249 consensus_queue.cc:260] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09120f3b2e7420bac607e5098631ea7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34129 } } peers { permanent_uuid: "ffc09e5c60f241c0943744cce3522b65" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33837 } } peers { permanent_uuid: "1a8acf9b759741e3916573c40fa373c0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45847 } }
I20260809 03:39:17.449410 32223 heartbeater.cc:503] Master 127.1.224.126:44523 was elected leader, sending a full tablet report...
I20260809 03:39:17.449915 31798 heartbeater.cc:503] Master 127.1.224.126:44523 was elected leader, sending a full tablet report...
I20260809 03:39:17.451285 32249 ts_tablet_manager.cc:1428] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 03:39:17.452886 32251 raft_consensus.cc:348] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09120f3b2e7420bac607e5098631ea7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34129 } } peers { permanent_uuid: "ffc09e5c60f241c0943744cce3522b65" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33837 } } peers { permanent_uuid: "1a8acf9b759741e3916573c40fa373c0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45847 } }
I20260809 03:39:17.453188 32251 raft_consensus.cc:374] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:17.453254 32251 raft_consensus.cc:729] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc09e5c60f241c0943744cce3522b65, State: Initialized, Role: FOLLOWER
I20260809 03:39:17.453739 32251 consensus_queue.cc:260] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09120f3b2e7420bac607e5098631ea7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34129 } } peers { permanent_uuid: "ffc09e5c60f241c0943744cce3522b65" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33837 } } peers { permanent_uuid: "1a8acf9b759741e3916573c40fa373c0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45847 } }
I20260809 03:39:17.454535 31999 heartbeater.cc:503] Master 127.1.224.126:44523 was elected leader, sending a full tablet report...
I20260809 03:39:17.454816 32251 ts_tablet_manager.cc:1428] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 03:39:17.505100 32262 raft_consensus.cc:482] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:39:17.505328 32262 raft_consensus.cc:504] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09120f3b2e7420bac607e5098631ea7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34129 } } peers { permanent_uuid: "ffc09e5c60f241c0943744cce3522b65" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33837 } } peers { permanent_uuid: "1a8acf9b759741e3916573c40fa373c0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45847 } }
I20260809 03:39:17.506697 32262 leader_election.cc:288] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c09120f3b2e7420bac607e5098631ea7 (127.1.224.65:34129), ffc09e5c60f241c0943744cce3522b65 (127.1.224.66:33837)
I20260809 03:39:17.511619 32244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.507142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:38019 (local address 127.1.224.65:34129)
0809 03:39:17.507273 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:17.507278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:17.507648 (+ 370us) server_negotiation.cc:407] Connection header received
0809 03:39:17.507762 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:17.507764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:17.507808 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:17.507906 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:17.508170 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.508909 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.510321 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.510959 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.511084 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.511141 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:17.511400 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 03:39:17.511477 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:39:17.511619 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.507123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:43251 (local address 127.1.224.66:33837)
0809 03:39:17.507292 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:17.507297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:17.507315 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:39:17.507379 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:17.507382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:17.507450 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:17.507523 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:17.507880 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.508701 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.510321 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.510946 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.511084 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.511141 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:17.511400 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 03:39:17.511477 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 03:39:17.511660 32225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.506962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:33837 (local address 127.1.224.67:43251)
0809 03:39:17.507114 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:17.507132 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:17.507263 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:17.507561 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:17.507567 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:17.507595 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:17.507761 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.507768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.508803 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.508805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.510168 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.510181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.511088 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.511092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.511236 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.511291 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 03:39:17.511371 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 03:39:17.511478 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":124,"spinlock_wait_cycles":30976}
I20260809 03:39:17.511668 32268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.506962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:34129 (local address 127.1.224.67:38019)
0809 03:39:17.507525 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:17.507541 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:17.507667 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:17.507935 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:17.507937 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:17.507954 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:17.508049 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.508053 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.509003 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.509006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.510202 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:17.510211 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.511088 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.511092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:17.511232 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.511288 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 03:39:17.511371 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 03:39:17.511478 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":218,"spinlock_wait_cycles":2304,"thread_start_us":149,"threads_started":1}
I20260809 03:39:17.512511 31737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb53dc0f708c4189b4daba0ac3de56a3" candidate_uuid: "1a8acf9b759741e3916573c40fa373c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c09120f3b2e7420bac607e5098631ea7" is_pre_election: true
I20260809 03:39:17.512508 31936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb53dc0f708c4189b4daba0ac3de56a3" candidate_uuid: "1a8acf9b759741e3916573c40fa373c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffc09e5c60f241c0943744cce3522b65" is_pre_election: true
I20260809 03:39:17.512887 31936 raft_consensus.cc:2454] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a8acf9b759741e3916573c40fa373c0 in term 0.
I20260809 03:39:17.512887 31737 raft_consensus.cc:2454] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a8acf9b759741e3916573c40fa373c0 in term 0.
I20260809 03:39:17.513460 32033 leader_election.cc:302] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a8acf9b759741e3916573c40fa373c0, c09120f3b2e7420bac607e5098631ea7, ffc09e5c60f241c0943744cce3522b65; no voters:
I20260809 03:39:17.513782 32262 raft_consensus.cc:2781] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:39:17.513885 32262 raft_consensus.cc:482] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:39:17.513944 32262 raft_consensus.cc:3037] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:17.515060 32262 raft_consensus.cc:504] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09120f3b2e7420bac607e5098631ea7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34129 } } peers { permanent_uuid: "ffc09e5c60f241c0943744cce3522b65" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33837 } } peers { permanent_uuid: "1a8acf9b759741e3916573c40fa373c0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45847 } }
I20260809 03:39:17.515496 32262 leader_election.cc:288] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [CANDIDATE]: Term 1 election: Requested vote from peers c09120f3b2e7420bac607e5098631ea7 (127.1.224.65:34129), ffc09e5c60f241c0943744cce3522b65 (127.1.224.66:33837)
I20260809 03:39:17.515877 31936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb53dc0f708c4189b4daba0ac3de56a3" candidate_uuid: "1a8acf9b759741e3916573c40fa373c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffc09e5c60f241c0943744cce3522b65"
I20260809 03:39:17.516018 31936 raft_consensus.cc:3037] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:17.516065 31737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb53dc0f708c4189b4daba0ac3de56a3" candidate_uuid: "1a8acf9b759741e3916573c40fa373c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c09120f3b2e7420bac607e5098631ea7"
I20260809 03:39:17.516214 31737 raft_consensus.cc:3037] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:17.517223 31936 raft_consensus.cc:2454] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a8acf9b759741e3916573c40fa373c0 in term 1.
I20260809 03:39:17.517370 31737 raft_consensus.cc:2454] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a8acf9b759741e3916573c40fa373c0 in term 1.
I20260809 03:39:17.517575 32040 leader_election.cc:302] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a8acf9b759741e3916573c40fa373c0, ffc09e5c60f241c0943744cce3522b65; no voters:
I20260809 03:39:17.517762 32262 raft_consensus.cc:2781] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:17.518003 32262 raft_consensus.cc:686] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 1 LEADER]: Becoming Leader. State: Replica: 1a8acf9b759741e3916573c40fa373c0, State: Running, Role: LEADER
I20260809 03:39:17.518373 32262 consensus_queue.cc:237] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09120f3b2e7420bac607e5098631ea7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34129 } } peers { permanent_uuid: "ffc09e5c60f241c0943744cce3522b65" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33837 } } peers { permanent_uuid: "1a8acf9b759741e3916573c40fa373c0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45847 } }
I20260809 03:39:17.521620 31519 catalog_manager.cc:5495] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a8acf9b759741e3916573c40fa373c0 (127.1.224.67). New cstate: current_term: 1 leader_uuid: "1a8acf9b759741e3916573c40fa373c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c09120f3b2e7420bac607e5098631ea7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffc09e5c60f241c0943744cce3522b65" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a8acf9b759741e3916573c40fa373c0" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 45847 } health_report { overall_health: HEALTHY } } }
I20260809 03:39:17.549007 32248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.545123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52806 (local address 127.1.224.67:45847)
0809 03:39:17.545276 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:17.545281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:17.545297 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:39:17.545414 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:17.545426 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:17.545484 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:17.545598 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:17.545859 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.546379 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.547367 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.547600 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.547714 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.548091 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:17.548585 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:17.548723 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:17.548788 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 03:39:17.548849 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:39:17.555142 31936 raft_consensus.cc:1261] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65 [term 1 FOLLOWER]: Refusing update from remote peer 1a8acf9b759741e3916573c40fa373c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:17.555142 31737 raft_consensus.cc:1261] T eb53dc0f708c4189b4daba0ac3de56a3 P c09120f3b2e7420bac607e5098631ea7 [term 1 FOLLOWER]: Refusing update from remote peer 1a8acf9b759741e3916573c40fa373c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:17.555923 32262 consensus_queue.cc:1035] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffc09e5c60f241c0943744cce3522b65" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 33837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:17.556160 32271 consensus_queue.cc:1035] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c09120f3b2e7420bac607e5098631ea7" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 34129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:17.565461 32279 mvcc.cc:205] Tried to move back new op lower bound from 7316466718938275840 to 7316466718797996032. Current Snapshot: MvccSnapshot[applied={T|T < 7316466718938275840}]
I20260809 03:39:17.567497 32277 mvcc.cc:205] Tried to move back new op lower bound from 7316466718938275840 to 7316466718797996032. Current Snapshot: MvccSnapshot[applied={T|T < 7316466718938275840}]
I20260809 03:39:17.575038 31519 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:39:17.575268 31519 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:39:17.578711 31519 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18daf6548696486ba4264eac5ce7be65: Sending DeleteTablet for 3 replicas of tablet eb53dc0f708c4189b4daba0ac3de56a3
I20260809 03:39:17.579690 31908 tablet_service.cc:1514] Processing DeleteTablet for tablet eb53dc0f708c4189b4daba0ac3de56a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:17 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:37016
I20260809 03:39:17.579701 31708 tablet_service.cc:1514] Processing DeleteTablet for tablet eb53dc0f708c4189b4daba0ac3de56a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:17 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:38588
I20260809 03:39:17.580209 32288 tablet_replica.cc:330] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65: stopping tablet replica
I20260809 03:39:17.580701 32288 raft_consensus.cc:2229] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:39:17.580729 32115 tablet_service.cc:1514] Processing DeleteTablet for tablet eb53dc0f708c4189b4daba0ac3de56a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:17 UTC) from {username='kudu', principal='kudu/127.1.224.126@KRBTEST.COM'} at 127.0.0.1:52792
I20260809 03:39:17.581241 32288 raft_consensus.cc:2258] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:39:17.582190 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 31612
I20260809 03:39:17.599068 32288 ts_tablet_manager.cc:1905] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:39:17.601694 32288 ts_tablet_manager.cc:1918] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:39:17.601833 32288 log.cc:1196] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/eb53dc0f708c4189b4daba0ac3de56a3
I20260809 03:39:17.602198 32288 ts_tablet_manager.cc:1939] T eb53dc0f708c4189b4daba0ac3de56a3 P ffc09e5c60f241c0943744cce3522b65: Deleting consensus metadata
I20260809 03:39:17.602591 32292 tablet_replica.cc:330] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0: stopping tablet replica
I20260809 03:39:17.603047 32292 raft_consensus.cc:2229] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:39:17.603549 32292 raft_consensus.cc:2258] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:39:17.603754 31499 catalog_manager.cc:4841] TS ffc09e5c60f241c0943744cce3522b65 (127.1.224.66:33837): tablet eb53dc0f708c4189b4daba0ac3de56a3 (table test-table [id=c2e24af21c544363a515454f5be2e8d8]) successfully deleted
I20260809 03:39:17.605317 32292 ts_tablet_manager.cc:1905] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:39:17.608278 32292 ts_tablet_manager.cc:1918] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260809 03:39:17.608290 31494 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:34129 (error 108)
I20260809 03:39:17.608650 32292 log.cc:1196] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/eb53dc0f708c4189b4daba0ac3de56a3
I20260809 03:39:17.609153 32292 ts_tablet_manager.cc:1939] T eb53dc0f708c4189b4daba0ac3de56a3 P 1a8acf9b759741e3916573c40fa373c0: Deleting consensus metadata
I20260809 03:39:17.609947 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 31805
I20260809 03:39:17.609965 32240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.609648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:34129 (local address 127.0.0.1:38592)
0809 03:39:17.609786 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:17.609872 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:34129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 03:39:17.610767 31497 catalog_manager.cc:4841] TS 1a8acf9b759741e3916573c40fa373c0 (127.1.224.67:45847): tablet eb53dc0f708c4189b4daba0ac3de56a3 (table test-table [id=c2e24af21c544363a515454f5be2e8d8]) successfully deleted
W20260809 03:39:17.624342 31494 catalog_manager.cc:4584] TS c09120f3b2e7420bac607e5098631ea7 (127.1.224.65:34129): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb53dc0f708c4189b4daba0ac3de56a3: Network error: Client connection negotiation failed: client connection to 127.1.224.65:34129: connect: Connection refused (error 111)
I20260809 03:39:17.631242 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 32006
I20260809 03:39:17.645524 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 31472
2026-08-09T03:39:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2676 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T03:39:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:17Z Disabled control of system clock
I20260809 03:39:17.689064 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:38027
--webserver_interface=127.1.224.126
--webserver_port=0
--builtin_ntp_servers=127.1.224.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:38027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:39:17.814349 32300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:17.814689 32300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:17.814729 32300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:17.819033 32300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:17.819118 32300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:17.819134 32300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:17.819151 32300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:17.819173 32300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:17.824697 32300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:38027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:38027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:17.825958 32300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:17.827443 32300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:17.837037 32308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:17.837138 32300 server_base.cc:1034] running on GCE node
W20260809 03:39:17.837031 32312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:17.837112 32309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:17.838047 32300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:17.839327 32300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:17.840540 32300 hybrid_clock.cc:648] HybridClock initialized: now 1786246757840520 us; error 33 us; skew 500 ppm
I20260809 03:39:17.842547 32300 webserver.cc:460] Webserver started at http://127.1.224.126:46241/ using document root <none> and password file <none>
I20260809 03:39:17.843182 32300 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:17.843251 32300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:17.843521 32300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:17.845817 32300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "979919a361f845319b9088ed0fbe0ce2"
format_stamp: "Formatted at 2026-08-09 03:39:17 on dist-test-slave-jq4t"
server_key: "4c0198abba32cd5d969bae372717915b"
server_key_iv: "096ede8278437cd2b21ecdb5a8d89c49"
server_key_version: "encryptionkey@0"
I20260809 03:39:17.846444 32300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "979919a361f845319b9088ed0fbe0ce2"
format_stamp: "Formatted at 2026-08-09 03:39:17 on dist-test-slave-jq4t"
server_key: "4c0198abba32cd5d969bae372717915b"
server_key_iv: "096ede8278437cd2b21ecdb5a8d89c49"
server_key_version: "encryptionkey@0"
I20260809 03:39:17.850690 32300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:39:17.853945 32323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:17.855549 32300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:39:17.855775 32300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "979919a361f845319b9088ed0fbe0ce2"
format_stamp: "Formatted at 2026-08-09 03:39:17 on dist-test-slave-jq4t"
server_key: "4c0198abba32cd5d969bae372717915b"
server_key_iv: "096ede8278437cd2b21ecdb5a8d89c49"
server_key_version: "encryptionkey@0"
I20260809 03:39:17.855948 32300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:17.891106 32300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:17.892033 32300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:17.892280 32300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:17.900370 32300 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:38027
I20260809 03:39:17.900401 32408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:38027 every 8 connection(s)
I20260809 03:39:17.901471 32300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:17.904861 32409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:17.906456 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 32300
I20260809 03:39:17.906615 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:17.906935 1921 external_mini_cluster.cc:1428] Setting key 662bb2819018e777bcb1841d0d3dbb71
I20260809 03:39:17.912272 32409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2: Bootstrap starting.
I20260809 03:39:17.915056 32409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:17.916054 32409 log.cc:824] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:17.918097 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:17.908535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58990 (local address 127.1.224.126:38027)
0809 03:39:17.909283 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:17.909304 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:17.909351 (+ 47us) server_negotiation.cc:407] Connection header received
0809 03:39:17.910264 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:17.910295 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:17.910816 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:17.911244 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:17.911703 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.913240 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.914141 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:17.914826 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:17.915014 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:17.915885 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:17.915893 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:17.915913 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:17.916452 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:17.916550 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:17.916585 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:17.916666 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:17.916831 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 03:39:17.917152 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":116,"threads_started":1}
I20260809 03:39:17.918632 32409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2: No bootstrap required, opened a new log
I20260809 03:39:17.922559 32409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979919a361f845319b9088ed0fbe0ce2" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 38027 } }
I20260809 03:39:17.922781 32409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:17.922816 32409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979919a361f845319b9088ed0fbe0ce2, State: Initialized, Role: FOLLOWER
I20260809 03:39:17.923244 32409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979919a361f845319b9088ed0fbe0ce2" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 38027 } }
I20260809 03:39:17.923375 32409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:17.923422 32409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:17.923504 32409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:17.924713 32409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979919a361f845319b9088ed0fbe0ce2" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 38027 } }
I20260809 03:39:17.925076 32409 leader_election.cc:302] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 979919a361f845319b9088ed0fbe0ce2; no voters:
I20260809 03:39:17.925422 32409 leader_election.cc:288] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:17.925552 32418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:17.925829 32418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [term 1 LEADER]: Becoming Leader. State: Replica: 979919a361f845319b9088ed0fbe0ce2, State: Running, Role: LEADER
I20260809 03:39:17.926139 32418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979919a361f845319b9088ed0fbe0ce2" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 38027 } }
I20260809 03:39:17.926725 32409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:17.927939 32421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 979919a361f845319b9088ed0fbe0ce2. Latest consensus state: current_term: 1 leader_uuid: "979919a361f845319b9088ed0fbe0ce2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979919a361f845319b9088ed0fbe0ce2" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 38027 } } }
I20260809 03:39:17.927933 32419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "979919a361f845319b9088ed0fbe0ce2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "979919a361f845319b9088ed0fbe0ce2" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 38027 } } }
I20260809 03:39:17.928277 32421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:17.928282 32419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:17.928738 32435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:17.932595 32435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:17.938908 32435 catalog_manager.cc:1347] Generated new cluster ID: 1ffb18fa1b764847b02eadbd6260f2ca
I20260809 03:39:17.939010 32435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:17.956562 32435 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:17.957597 32435 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:17.964128 32435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2: Generated new TSK 0
I20260809 03:39:17.964936 32435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:17.974145 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.65:0
--local_ip_for_outbound_sockets=127.1.224.65
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:38027
--builtin_ntp_servers=127.1.224.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:39:18.094779 32447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:18.095117 32447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:18.095161 32447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:18.100056 32447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:18.100137 32447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:18.100230 32447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.65
I20260809 03:39:18.105336 32447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.224.65
--webserver_port=0
--tserver_master_addrs=127.1.224.126:38027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.65
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:18.106689 32447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:18.108294 32447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:18.119169 32458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:18.119174 32462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:18.119205 32456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:18.119290 32447 server_base.cc:1034] running on GCE node
I20260809 03:39:18.119940 32447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:18.120760 32447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:18.121944 32447 hybrid_clock.cc:648] HybridClock initialized: now 1786246758121927 us; error 30 us; skew 500 ppm
I20260809 03:39:18.124099 32447 webserver.cc:460] Webserver started at http://127.1.224.65:34851/ using document root <none> and password file <none>
I20260809 03:39:18.124836 32447 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:18.124945 32447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:18.125195 32447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:18.127637 32447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/data/instance:
uuid: "09d7f8076ddb4f13a5d4b625eba78972"
format_stamp: "Formatted at 2026-08-09 03:39:18 on dist-test-slave-jq4t"
server_key: "95f329add6e55f0009e262d60902a2e0"
server_key_iv: "6452f86fc313be6a3ee1095cf502f84c"
server_key_version: "encryptionkey@0"
I20260809 03:39:18.128247 32447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance:
uuid: "09d7f8076ddb4f13a5d4b625eba78972"
format_stamp: "Formatted at 2026-08-09 03:39:18 on dist-test-slave-jq4t"
server_key: "95f329add6e55f0009e262d60902a2e0"
server_key_iv: "6452f86fc313be6a3ee1095cf502f84c"
server_key_version: "encryptionkey@0"
I20260809 03:39:18.132700 32447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 03:39:18.135511 32472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:18.136920 32447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:18.137070 32447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/wal
uuid: "09d7f8076ddb4f13a5d4b625eba78972"
format_stamp: "Formatted at 2026-08-09 03:39:18 on dist-test-slave-jq4t"
server_key: "95f329add6e55f0009e262d60902a2e0"
server_key_iv: "6452f86fc313be6a3ee1095cf502f84c"
server_key_version: "encryptionkey@0"
I20260809 03:39:18.137202 32447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:18.148324 32447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:18.149067 32447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:18.149266 32447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:18.149878 32447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:18.150885 32447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:18.150956 32447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:18.151031 32447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:18.151082 32447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:18.160089 32447 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.65:37601
I20260809 03:39:18.160115 32639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.65:37601 every 8 connection(s)
I20260809 03:39:18.161139 32447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/data/info.pb
I20260809 03:39:18.162706 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 32447
I20260809 03:39:18.162833 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-0/wal/instance
I20260809 03:39:18.163204 1921 external_mini_cluster.cc:1428] Setting key bfd90387fccf752a23c848fc232888ca
I20260809 03:39:18.165575 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.66:0
--local_ip_for_outbound_sockets=127.1.224.66
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:38027
--builtin_ntp_servers=127.1.224.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:39:18.173053 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.163319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.65:49745 (local address 127.1.224.126:38027)
0809 03:39:18.163484 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:18.163488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:18.164820 (+ 1332us) server_negotiation.cc:407] Connection header received
0809 03:39:18.166966 (+ 2146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:18.166969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:18.167023 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:18.167105 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:18.168269 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.169189 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.170440 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.170871 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.171015 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.172316 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:18.172324 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:18.172330 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:18.172352 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:18.172381 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:18.172395 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:18.172535 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:18.172726 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 03:39:18.172827 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 03:39:18.173913 32646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.163823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:38027 (local address 127.1.224.65:49745)
0809 03:39:18.164645 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:18.164701 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:18.166644 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:18.167336 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:18.167357 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:18.167484 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:18.168097 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.168112 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.169333 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.169336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.170286 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.170297 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.171069 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.171072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.171244 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.171995 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:39:18.171997 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:18.172043 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:39:18.172127 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:18.172557 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:18.172566 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:18.172571 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:39:18.172681 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 03:39:18.172956 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":145,"threads_started":1}
I20260809 03:39:18.175385 32640 heartbeater.cc:347] Connected to a master server at 127.1.224.126:38027
I20260809 03:39:18.175707 32640 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:18.176534 32640 heartbeater.cc:511] Master 127.1.224.126:38027 requested a full tablet report, sending...
I20260809 03:39:18.178368 32349 ts_manager.cc:194] Registered new tserver with Master: 09d7f8076ddb4f13a5d4b625eba78972 (127.1.224.65:37601)
I20260809 03:39:18.179833 32349 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.224.65:49745
W20260809 03:39:18.288502 32647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:18.288800 32647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:18.288873 32647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:18.293107 32647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:18.293191 32647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:18.293249 32647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.66
I20260809 03:39:18.298174 32647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.224.66
--webserver_port=0
--tserver_master_addrs=127.1.224.126:38027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.66
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:18.299436 32647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:18.300972 32647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:18.311146 32656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:18.311164 32657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:18.311160 32659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:18.311364 32647 server_base.cc:1034] running on GCE node
I20260809 03:39:18.311890 32647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:18.312644 32647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:18.313808 32647 hybrid_clock.cc:648] HybridClock initialized: now 1786246758313789 us; error 33 us; skew 500 ppm
I20260809 03:39:18.315845 32647 webserver.cc:460] Webserver started at http://127.1.224.66:40979/ using document root <none> and password file <none>
I20260809 03:39:18.316406 32647 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:18.316470 32647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:18.316740 32647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:18.318847 32647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/data/instance:
uuid: "29d3f5255dc2437baed0e82a1fd2b2e0"
format_stamp: "Formatted at 2026-08-09 03:39:18 on dist-test-slave-jq4t"
server_key: "396031c9c9168eaf50fc2d8a88368f52"
server_key_iv: "5783dbce86269b88c1e61dca26930907"
server_key_version: "encryptionkey@0"
I20260809 03:39:18.319371 32647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance:
uuid: "29d3f5255dc2437baed0e82a1fd2b2e0"
format_stamp: "Formatted at 2026-08-09 03:39:18 on dist-test-slave-jq4t"
server_key: "396031c9c9168eaf50fc2d8a88368f52"
server_key_iv: "5783dbce86269b88c1e61dca26930907"
server_key_version: "encryptionkey@0"
I20260809 03:39:18.323114 32647 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 03:39:18.325585 32672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:18.326813 32647 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:39:18.326946 32647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/wal
uuid: "29d3f5255dc2437baed0e82a1fd2b2e0"
format_stamp: "Formatted at 2026-08-09 03:39:18 on dist-test-slave-jq4t"
server_key: "396031c9c9168eaf50fc2d8a88368f52"
server_key_iv: "5783dbce86269b88c1e61dca26930907"
server_key_version: "encryptionkey@0"
I20260809 03:39:18.327066 32647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:18.342465 32647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:18.343217 32647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:18.343425 32647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:18.344053 32647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:18.345125 32647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:18.345206 32647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:18.345281 32647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:18.345331 32647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:18.354471 32647 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.66:32791
I20260809 03:39:18.354548 397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.66:32791 every 8 connection(s)
I20260809 03:39:18.355444 32647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/data/info.pb
I20260809 03:39:18.364758 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 32647
I20260809 03:39:18.364910 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/wal/instance
I20260809 03:39:18.364993 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.357476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:38121 (local address 127.1.224.126:38027)
0809 03:39:18.357634 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:18.357638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:18.358707 (+ 1069us) server_negotiation.cc:407] Connection header received
0809 03:39:18.360125 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:18.360128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:18.360180 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:18.360256 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:18.361216 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.361791 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.362958 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.363176 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.363255 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.364364 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:18.364367 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:18.364369 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:18.364383 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:18.364403 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:18.364412 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:18.364543 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:18.364733 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 03:39:18.364796 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 03:39:18.365236 1921 external_mini_cluster.cc:1428] Setting key 134a1be3e33ca4857ad607a0a21ca578
I20260809 03:39:18.366021 402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.357874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:38027 (local address 127.1.224.66:38121)
0809 03:39:18.358556 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:18.358597 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:18.359924 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:18.360409 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:18.360416 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:18.360520 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:18.361044 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.361058 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.361931 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.361934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.362835 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.362843 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.363318 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.363321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.363465 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.364066 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:39:18.364068 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:18.364115 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:39:18.364194 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:18.364542 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:18.364555 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:18.364564 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 03:39:18.364714 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 03:39:18.365060 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":120,"threads_started":1}
I20260809 03:39:18.367241 399 heartbeater.cc:347] Connected to a master server at 127.1.224.126:38027
I20260809 03:39:18.367507 399 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:18.367581 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.224.67:0
--local_ip_for_outbound_sockets=127.1.224.67
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:38027
--builtin_ntp_servers=127.1.224.84:35215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:39:18.368166 399 heartbeater.cc:511] Master 127.1.224.126:38027 requested a full tablet report, sending...
I20260809 03:39:18.369376 32349 ts_manager.cc:194] Registered new tserver with Master: 29d3f5255dc2437baed0e82a1fd2b2e0 (127.1.224.66:32791)
I20260809 03:39:18.370012 32349 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.224.66:38121
W20260809 03:39:18.489208 403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:18.489563 403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:18.489639 403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:18.493973 403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:18.494082 403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:18.494186 403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.224.67
I20260809 03:39:18.499297 403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.224.67
--webserver_port=0
--tserver_master_addrs=127.1.224.126:38027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.224.67
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:18.500615 403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:18.502107 403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:18.512014 410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:18.511999 414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:18.512027 412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:18.512182 403 server_base.cc:1034] running on GCE node
I20260809 03:39:18.512830 403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:18.513512 403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:18.514703 403 hybrid_clock.cc:648] HybridClock initialized: now 1786246758514687 us; error 28 us; skew 500 ppm
I20260809 03:39:18.516800 403 webserver.cc:460] Webserver started at http://127.1.224.67:39975/ using document root <none> and password file <none>
I20260809 03:39:18.517433 403 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:18.517531 403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:18.517786 403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:18.519989 403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/data/instance:
uuid: "b74600bf5dbf4262ba38f7cea662c1ed"
format_stamp: "Formatted at 2026-08-09 03:39:18 on dist-test-slave-jq4t"
server_key: "1edae9a7a8adf26f721fd5d145f23964"
server_key_iv: "6e2b214c1ea65e918227f2e349784963"
server_key_version: "encryptionkey@0"
I20260809 03:39:18.520665 403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance:
uuid: "b74600bf5dbf4262ba38f7cea662c1ed"
format_stamp: "Formatted at 2026-08-09 03:39:18 on dist-test-slave-jq4t"
server_key: "1edae9a7a8adf26f721fd5d145f23964"
server_key_iv: "6e2b214c1ea65e918227f2e349784963"
server_key_version: "encryptionkey@0"
I20260809 03:39:18.524686 403 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:39:18.527271 420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:18.528640 403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:18.528785 403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/wal
uuid: "b74600bf5dbf4262ba38f7cea662c1ed"
format_stamp: "Formatted at 2026-08-09 03:39:18 on dist-test-slave-jq4t"
server_key: "1edae9a7a8adf26f721fd5d145f23964"
server_key_iv: "6e2b214c1ea65e918227f2e349784963"
server_key_version: "encryptionkey@0"
I20260809 03:39:18.528929 403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:18.539520 403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:18.540261 403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:18.540469 403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:18.541271 403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:39:18.542296 403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:39:18.542366 403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:18.542461 403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:39:18.542508 403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:18.551471 403 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.67:37673
I20260809 03:39:18.551506 610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.67:37673 every 8 connection(s)
I20260809 03:39:18.552402 403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/data/info.pb
I20260809 03:39:18.556315 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 403
I20260809 03:39:18.556468 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/wal/instance
I20260809 03:39:18.556862 1921 external_mini_cluster.cc:1428] Setting key 34f0c38d8287d8455835fffb6fd8134e
I20260809 03:39:18.562777 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.554375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.67:37639 (local address 127.1.224.126:38027)
0809 03:39:18.554528 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:18.554532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:18.555546 (+ 1014us) server_negotiation.cc:407] Connection header received
0809 03:39:18.557380 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:18.557387 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:18.557454 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:18.557549 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:18.558823 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.559486 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.560786 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.561019 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.561136 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.562254 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:18.562263 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:18.562265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:18.562284 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:18.562304 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:18.562312 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:18.562391 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:18.562561 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 03:39:18.562629 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:39:18.563683 613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.554744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:38027 (local address 127.1.224.67:37639)
0809 03:39:18.555416 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:18.555457 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:18.557136 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:18.557851 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:18.557858 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:18.557983 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:18.558683 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.558699 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.559617 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.559624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.560653 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.560665 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.561179 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.561183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.561362 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.561989 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:39:18.561991 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:18.562033 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:39:18.562097 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:18.562404 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:18.562416 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:18.562422 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:39:18.562559 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 03:39:18.562862 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":137,"threads_started":1}
I20260809 03:39:18.564859 611 heartbeater.cc:347] Connected to a master server at 127.1.224.126:38027
I20260809 03:39:18.565116 611 heartbeater.cc:464] Registering TS with master...
I20260809 03:39:18.565649 611 heartbeater.cc:511] Master 127.1.224.126:38027 requested a full tablet report, sending...
I20260809 03:39:18.566787 32349 ts_manager.cc:194] Registered new tserver with Master: b74600bf5dbf4262ba38f7cea662c1ed (127.1.224.67:37673)
I20260809 03:39:18.567375 32349 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.224.67:37639
I20260809 03:39:18.571233 1921 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:39:18.577427 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.573411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58996 (local address 127.1.224.126:38027)
0809 03:39:18.573586 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:18.573591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:18.573753 (+ 162us) server_negotiation.cc:407] Connection header received
0809 03:39:18.573940 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:18.573942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:18.573984 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:18.574067 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:18.574374 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.574911 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.575776 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.576000 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.576113 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.576956 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:18.576960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:18.576962 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:18.576982 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:18.577001 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:18.577016 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:18.577099 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:18.577219 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 03:39:18.577275 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 03:39:18.580627 32349 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:39:18.583743 32349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:39:18.598873 629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.592770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60634 (local address 127.1.224.66:32791)
0809 03:39:18.593418 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:18.593431 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:18.593940 (+ 509us) server_negotiation.cc:407] Connection header received
0809 03:39:18.594195 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:18.594203 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:18.594400 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:18.594547 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:18.595039 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.596169 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.597447 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.598057 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.598192 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.598421 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:18.598564 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 03:39:18.598687 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":126,"threads_started":1}
I20260809 03:39:18.599088 631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.593407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.1.224.67:37673)
0809 03:39:18.594135 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:18.594149 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:18.594173 (+ 24us) server_negotiation.cc:407] Connection header received
0809 03:39:18.594243 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:18.594247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:18.594391 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:18.594547 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:18.594750 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.595610 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.597379 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.598218 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.598335 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.598551 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:18.598754 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 03:39:18.598883 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":104,"threads_started":1}
I20260809 03:39:18.599319 630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.592762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:37673 (local address 127.0.0.1:44232)
0809 03:39:18.593792 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:18.593808 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:18.593900 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:18.594521 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:18.594524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:18.594541 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:18.594653 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.594659 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.595745 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.595749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.597226 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.597238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.598505 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.598512 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.598602 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.598625 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:39:18.598699 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 03:39:18.598833 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":1,"thread_start_us":109,"threads_started":1}
I20260809 03:39:18.599493 627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.592720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42950 (local address 127.1.224.65:37601)
0809 03:39:18.593647 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:18.593662 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:18.593692 (+ 30us) server_negotiation.cc:407] Connection header received
0809 03:39:18.593919 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:18.593926 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:18.594163 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:18.594329 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:18.594668 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.595880 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.597563 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.598326 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.598419 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.598568 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:18.599112 (+ 544us) server_negotiation.cc:299] Negotiation successful
0809 03:39:18.599302 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":813,"thread_start_us":125,"threads_started":1}
I20260809 03:39:18.599747 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.593081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.66:32791 (local address 127.0.0.1:60634)
0809 03:39:18.593846 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:18.593861 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:18.594006 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:18.594728 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:18.594732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:18.594750 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:18.594858 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.594864 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.596311 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.596315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.597285 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.597321 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.598193 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.598197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.598307 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.598346 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 03:39:18.599124 (+ 778us) client_negotiation.cc:240] Negotiation successful
0809 03:39:18.599271 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"mutex_wait_us":1,"thread_start_us":108,"threads_started":1}
I20260809 03:39:18.599774 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.592544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:37601 (local address 127.0.0.1:42950)
0809 03:39:18.593487 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:18.593548 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:18.593688 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:18.594317 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:18.594324 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:18.594347 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:18.594526 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.594534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.596167 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.596170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.597432 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.597442 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.598642 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.598645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.598753 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.598779 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 03:39:18.598851 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 03:39:18.598918 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
I20260809 03:39:18.602622 32749 tablet_service.cc:1467] Processing CreateTablet for tablet b55deba9817d487f896b279da45bfe40 (DEFAULT_TABLE table=test-table [id=9696243ddde54de59df33176d1f3902d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:18.602713 32542 tablet_service.cc:1467] Processing CreateTablet for tablet b55deba9817d487f896b279da45bfe40 (DEFAULT_TABLE table=test-table [id=9696243ddde54de59df33176d1f3902d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:18.602715 500 tablet_service.cc:1467] Processing CreateTablet for tablet b55deba9817d487f896b279da45bfe40 (DEFAULT_TABLE table=test-table [id=9696243ddde54de59df33176d1f3902d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:39:18.603837 32542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b55deba9817d487f896b279da45bfe40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:18.603837 500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b55deba9817d487f896b279da45bfe40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:18.603842 32749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b55deba9817d487f896b279da45bfe40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:18.611032 633 tablet_bootstrap.cc:492] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0: Bootstrap starting.
I20260809 03:39:18.612102 635 tablet_bootstrap.cc:492] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972: Bootstrap starting.
I20260809 03:39:18.612871 636 tablet_bootstrap.cc:492] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed: Bootstrap starting.
I20260809 03:39:18.613932 633 tablet_bootstrap.cc:654] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:18.614202 635 tablet_bootstrap.cc:654] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:18.614897 633 log.cc:824] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:18.615041 635 log.cc:824] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:18.615078 636 tablet_bootstrap.cc:654] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:18.615849 636 log.cc:824] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:18.617144 635 tablet_bootstrap.cc:492] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972: No bootstrap required, opened a new log
I20260809 03:39:18.617202 633 tablet_bootstrap.cc:492] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0: No bootstrap required, opened a new log
I20260809 03:39:18.617380 635 ts_tablet_manager.cc:1397] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20260809 03:39:18.617401 633 ts_tablet_manager.cc:1397] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 03:39:18.617858 636 tablet_bootstrap.cc:492] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed: No bootstrap required, opened a new log
I20260809 03:39:18.618062 636 ts_tablet_manager.cc:1397] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:39:18.620878 635 raft_consensus.cc:348] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d7f8076ddb4f13a5d4b625eba78972" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37601 } } peers { permanent_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 32791 } } peers { permanent_uuid: "b74600bf5dbf4262ba38f7cea662c1ed" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37673 } }
I20260809 03:39:18.621151 635 raft_consensus.cc:374] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:18.621205 635 raft_consensus.cc:729] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d7f8076ddb4f13a5d4b625eba78972, State: Initialized, Role: FOLLOWER
I20260809 03:39:18.621722 635 consensus_queue.cc:260] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d7f8076ddb4f13a5d4b625eba78972" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37601 } } peers { permanent_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 32791 } } peers { permanent_uuid: "b74600bf5dbf4262ba38f7cea662c1ed" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37673 } }
I20260809 03:39:18.621803 633 raft_consensus.cc:348] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d7f8076ddb4f13a5d4b625eba78972" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37601 } } peers { permanent_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 32791 } } peers { permanent_uuid: "b74600bf5dbf4262ba38f7cea662c1ed" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37673 } }
I20260809 03:39:18.622058 633 raft_consensus.cc:374] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:18.622109 633 raft_consensus.cc:729] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29d3f5255dc2437baed0e82a1fd2b2e0, State: Initialized, Role: FOLLOWER
I20260809 03:39:18.622241 636 raft_consensus.cc:348] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d7f8076ddb4f13a5d4b625eba78972" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37601 } } peers { permanent_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 32791 } } peers { permanent_uuid: "b74600bf5dbf4262ba38f7cea662c1ed" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37673 } }
I20260809 03:39:18.622462 636 raft_consensus.cc:374] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:18.622515 636 raft_consensus.cc:729] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b74600bf5dbf4262ba38f7cea662c1ed, State: Initialized, Role: FOLLOWER
I20260809 03:39:18.622577 633 consensus_queue.cc:260] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d7f8076ddb4f13a5d4b625eba78972" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37601 } } peers { permanent_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 32791 } } peers { permanent_uuid: "b74600bf5dbf4262ba38f7cea662c1ed" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37673 } }
I20260809 03:39:18.622869 635 ts_tablet_manager.cc:1428] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260809 03:39:18.622951 636 consensus_queue.cc:260] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d7f8076ddb4f13a5d4b625eba78972" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37601 } } peers { permanent_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 32791 } } peers { permanent_uuid: "b74600bf5dbf4262ba38f7cea662c1ed" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37673 } }
I20260809 03:39:18.623303 32640 heartbeater.cc:503] Master 127.1.224.126:38027 was elected leader, sending a full tablet report...
I20260809 03:39:18.623674 611 heartbeater.cc:503] Master 127.1.224.126:38027 was elected leader, sending a full tablet report...
I20260809 03:39:18.623847 636 ts_tablet_manager.cc:1428] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 03:39:18.623535 633 ts_tablet_manager.cc:1428] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260809 03:39:18.624286 399 heartbeater.cc:503] Master 127.1.224.126:38027 was elected leader, sending a full tablet report...
W20260809 03:39:18.662603 32644 tablet.cc:2369] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:39:18.711663 644 raft_consensus.cc:482] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:39:18.711956 644 raft_consensus.cc:504] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d7f8076ddb4f13a5d4b625eba78972" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37601 } } peers { permanent_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 32791 } } peers { permanent_uuid: "b74600bf5dbf4262ba38f7cea662c1ed" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37673 } }
I20260809 03:39:18.713337 644 leader_election.cc:288] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09d7f8076ddb4f13a5d4b625eba78972 (127.1.224.65:37601), b74600bf5dbf4262ba38f7cea662c1ed (127.1.224.67:37673)
I20260809 03:39:18.718108 402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.713684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:37601 (local address 127.1.224.66:59779)
0809 03:39:18.713853 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:18.713899 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:18.714027 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:18.714387 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:18.714390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:18.714404 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:18.714556 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.714562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.715588 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.715595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.716837 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.716849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.717713 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.717716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.717791 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.717813 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 03:39:18.717877 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 03:39:18.717937 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 03:39:18.718160 627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.713792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:59779 (local address 127.1.224.65:37601)
0809 03:39:18.713943 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:18.713951 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:18.713970 (+ 19us) server_negotiation.cc:407] Connection header received
0809 03:39:18.714207 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:18.714210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:18.714263 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:18.714361 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:18.714678 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.715462 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.717011 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.717581 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.717710 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.717765 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:18.717920 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 03:39:18.718015 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:39:18.718487 631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.713920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.224.66:43425 (local address 127.1.224.67:37673)
0809 03:39:18.714079 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:18.714083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:18.714357 (+ 274us) server_negotiation.cc:407] Connection header received
0809 03:39:18.714519 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:18.714521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:18.714562 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:18.714655 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:39:18.715178 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.715870 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.717253 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.717889 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.718008 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.718051 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:18.718281 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 03:39:18.718393 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:39:18.718869 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.713684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.67:37673 (local address 127.1.224.66:43425)
0809 03:39:18.714254 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:18.714266 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:18.714396 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:18.714896 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:18.714899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:18.714918 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:39:18.715027 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.715056 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.715987 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.715990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.717128 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:18.717136 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.718019 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.718023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:18.718108 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.718140 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 03:39:18.718656 (+ 516us) client_negotiation.cc:240] Negotiation successful
0809 03:39:18.718734 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":168,"thread_start_us":160,"threads_started":1}
I20260809 03:39:18.718894 32575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b55deba9817d487f896b279da45bfe40" candidate_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d7f8076ddb4f13a5d4b625eba78972" is_pre_election: true
I20260809 03:39:18.719252 32575 raft_consensus.cc:2454] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29d3f5255dc2437baed0e82a1fd2b2e0 in term 0.
I20260809 03:39:18.719589 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b55deba9817d487f896b279da45bfe40" candidate_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b74600bf5dbf4262ba38f7cea662c1ed" is_pre_election: true
I20260809 03:39:18.719842 32679 leader_election.cc:302] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d7f8076ddb4f13a5d4b625eba78972, 29d3f5255dc2437baed0e82a1fd2b2e0; no voters:
I20260809 03:39:18.719911 530 raft_consensus.cc:2454] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29d3f5255dc2437baed0e82a1fd2b2e0 in term 0.
I20260809 03:39:18.720192 644 raft_consensus.cc:2781] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:39:18.720304 644 raft_consensus.cc:482] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:39:18.720364 644 raft_consensus.cc:3037] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:18.721483 644 raft_consensus.cc:504] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d7f8076ddb4f13a5d4b625eba78972" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37601 } } peers { permanent_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 32791 } } peers { permanent_uuid: "b74600bf5dbf4262ba38f7cea662c1ed" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37673 } }
I20260809 03:39:18.721860 644 leader_election.cc:288] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [CANDIDATE]: Term 1 election: Requested vote from peers 09d7f8076ddb4f13a5d4b625eba78972 (127.1.224.65:37601), b74600bf5dbf4262ba38f7cea662c1ed (127.1.224.67:37673)
I20260809 03:39:18.722234 530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b55deba9817d487f896b279da45bfe40" candidate_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b74600bf5dbf4262ba38f7cea662c1ed"
I20260809 03:39:18.722234 32575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b55deba9817d487f896b279da45bfe40" candidate_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d7f8076ddb4f13a5d4b625eba78972"
I20260809 03:39:18.722363 32575 raft_consensus.cc:3037] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:18.722363 530 raft_consensus.cc:3037] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:18.723688 530 raft_consensus.cc:2454] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29d3f5255dc2437baed0e82a1fd2b2e0 in term 1.
I20260809 03:39:18.723688 32575 raft_consensus.cc:2454] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29d3f5255dc2437baed0e82a1fd2b2e0 in term 1.
I20260809 03:39:18.724049 32675 leader_election.cc:302] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 09d7f8076ddb4f13a5d4b625eba78972, 29d3f5255dc2437baed0e82a1fd2b2e0, b74600bf5dbf4262ba38f7cea662c1ed; no voters:
I20260809 03:39:18.724246 644 raft_consensus.cc:2781] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:18.724560 644 raft_consensus.cc:686] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 1 LEADER]: Becoming Leader. State: Replica: 29d3f5255dc2437baed0e82a1fd2b2e0, State: Running, Role: LEADER
I20260809 03:39:18.724923 644 consensus_queue.cc:237] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d7f8076ddb4f13a5d4b625eba78972" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37601 } } peers { permanent_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 32791 } } peers { permanent_uuid: "b74600bf5dbf4262ba38f7cea662c1ed" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37673 } }
I20260809 03:39:18.728626 32347 catalog_manager.cc:5495] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29d3f5255dc2437baed0e82a1fd2b2e0 (127.1.224.66). New cstate: current_term: 1 leader_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d7f8076ddb4f13a5d4b625eba78972" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29d3f5255dc2437baed0e82a1fd2b2e0" member_type: VOTER last_known_addr { host: "127.1.224.66" port: 32791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b74600bf5dbf4262ba38f7cea662c1ed" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37673 } health_report { overall_health: UNKNOWN } } }
I20260809 03:39:18.754725 629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.751268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.1.224.66:32791)
0809 03:39:18.751409 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:18.751413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:18.751428 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:39:18.751522 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:18.751524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:18.751585 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:18.751677 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:18.751896 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.752401 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.753290 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:18.753514 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:18.753594 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:18.753863 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:18.754310 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:18.754403 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:18.754465 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 03:39:18.754514 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:39:18.761116 530 raft_consensus.cc:1261] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed [term 1 FOLLOWER]: Refusing update from remote peer 29d3f5255dc2437baed0e82a1fd2b2e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:18.761116 32575 raft_consensus.cc:1261] T b55deba9817d487f896b279da45bfe40 P 09d7f8076ddb4f13a5d4b625eba78972 [term 1 FOLLOWER]: Refusing update from remote peer 29d3f5255dc2437baed0e82a1fd2b2e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:18.761973 648 consensus_queue.cc:1035] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d7f8076ddb4f13a5d4b625eba78972" member_type: VOTER last_known_addr { host: "127.1.224.65" port: 37601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:18.762187 644 consensus_queue.cc:1035] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b74600bf5dbf4262ba38f7cea662c1ed" member_type: VOTER last_known_addr { host: "127.1.224.67" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:18.769528 652 mvcc.cc:205] Tried to move back new op lower bound from 7316466723876880384 to 7316466723740381184. Current Snapshot: MvccSnapshot[applied={T|T < 7316466723876880384}]
I20260809 03:39:18.772069 655 mvcc.cc:205] Tried to move back new op lower bound from 7316466723876880384 to 7316466723740381184. Current Snapshot: MvccSnapshot[applied={T|T < 7316466723876880384}]
I20260809 03:39:18.772403 653 mvcc.cc:205] Tried to move back new op lower bound from 7316466723876880384 to 7316466723740381184. Current Snapshot: MvccSnapshot[applied={T|T < 7316466723876880384}]
I20260809 03:39:18.779384 32347 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:39:18.779610 32347 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:39:18.782615 32347 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 979919a361f845319b9088ed0fbe0ce2: Sending DeleteTablet for 3 replicas of tablet b55deba9817d487f896b279da45bfe40
I20260809 03:39:18.783595 32542 tablet_service.cc:1514] Processing DeleteTablet for tablet b55deba9817d487f896b279da45bfe40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:18 UTC) from {username='slave'} at 127.0.0.1:42950
I20260809 03:39:18.783591 32749 tablet_service.cc:1514] Processing DeleteTablet for tablet b55deba9817d487f896b279da45bfe40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:18 UTC) from {username='slave'} at 127.0.0.1:60634
I20260809 03:39:18.783767 500 tablet_service.cc:1514] Processing DeleteTablet for tablet b55deba9817d487f896b279da45bfe40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:39:18 UTC) from {username='slave'} at 127.0.0.1:44232
I20260809 03:39:18.784528 667 tablet_replica.cc:330] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0: stopping tablet replica
I20260809 03:39:18.784924 667 raft_consensus.cc:2229] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:39:18.785068 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 32447
I20260809 03:39:18.785432 667 raft_consensus.cc:2258] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:39:18.785113 670 tablet_replica.cc:330] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed: stopping tablet replica
I20260809 03:39:18.789381 670 raft_consensus.cc:2229] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:39:18.789824 670 raft_consensus.cc:2258] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:39:18.791482 667 ts_tablet_manager.cc:1905] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:39:18.792917 670 ts_tablet_manager.cc:1905] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:39:18.794734 667 ts_tablet_manager.cc:1918] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:39:18.794899 667 log.cc:1196] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-1/wal/wals/b55deba9817d487f896b279da45bfe40
I20260809 03:39:18.795296 667 ts_tablet_manager.cc:1939] T b55deba9817d487f896b279da45bfe40 P 29d3f5255dc2437baed0e82a1fd2b2e0: Deleting consensus metadata
I20260809 03:39:18.796134 670 ts_tablet_manager.cc:1918] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:39:18.796296 670 log.cc:1196] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed: Deleting WAL directory at /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786246494448707-1921-0/minicluster-data/ts-2/wal/wals/b55deba9817d487f896b279da45bfe40
I20260809 03:39:18.796720 670 ts_tablet_manager.cc:1939] T b55deba9817d487f896b279da45bfe40 P b74600bf5dbf4262ba38f7cea662c1ed: Deleting consensus metadata
I20260809 03:39:18.797047 32324 catalog_manager.cc:4841] TS 29d3f5255dc2437baed0e82a1fd2b2e0 (127.1.224.66:32791): tablet b55deba9817d487f896b279da45bfe40 (table test-table [id=9696243ddde54de59df33176d1f3902d]) successfully deleted
I20260809 03:39:18.798126 32325 catalog_manager.cc:4841] TS b74600bf5dbf4262ba38f7cea662c1ed (127.1.224.67:37673): tablet b55deba9817d487f896b279da45bfe40 (table test-table [id=9696243ddde54de59df33176d1f3902d]) successfully deleted
W20260809 03:39:18.804750 32327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.224.65:37601 (error 108)
I20260809 03:39:18.804926 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 32647
I20260809 03:39:18.809311 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:18.808774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.65:37601 (local address 127.0.0.1:42956)
0809 03:39:18.809063 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:18.809173 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.65:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
W20260809 03:39:18.809744 32327 catalog_manager.cc:4584] TS 09d7f8076ddb4f13a5d4b625eba78972 (127.1.224.65:37601): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b55deba9817d487f896b279da45bfe40: Network error: Client connection negotiation failed: client connection to 127.1.224.65:37601: connect: Connection refused (error 111)
I20260809 03:39:18.823458 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 403
I20260809 03:39:18.834486 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 32300
2026-08-09T03:39:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1186 ms)
[----------] 6 tests from EncryptionPolicyTest (11909 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:18 dist-test-slave-jq4t krb5kdc[675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:18 dist-test-slave-jq4t krb5kdc[675](info): set up 2 sockets
Aug 09 03:39:18 dist-test-slave-jq4t krb5kdc[675](info): commencing operation
krb5kdc: starting...
W20260809 03:39:20.098059 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:20 dist-test-slave-jq4t krb5kdc[675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:39:20.235905 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32783
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:20.355019 698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:20.355391 698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:20.355472 698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:20.359580 698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:20.359680 698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:20.359719 698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:20.359781 698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:20.359820 698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:20.365265 698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32783
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:20.366446 698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:20.367558 698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:20.377036 710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:20.377027 706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:20.377027 708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:20.377367 698 server_base.cc:1034] running on GCE node
I20260809 03:39:20.378031 698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:20.379267 698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:20.380669 698 hybrid_clock.cc:648] HybridClock initialized: now 1786246760380636 us; error 46 us; skew 500 ppm
Aug 09 03:39:20 dist-test-slave-jq4t krb5kdc[675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246760, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:20.388199 698 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:39:20.389427 698 webserver.cc:460] Webserver started at http://127.0.0.1:45829/ using document root <none> and password file <none>
I20260809 03:39:20.390059 698 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:20.390131 698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:20.390328 698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:20.392724 698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "8f5ba61c0d384cc6b1c8c7d776b33230"
format_stamp: "Formatted at 2026-08-09 03:39:20 on dist-test-slave-jq4t"
server_key: "1088a89a35714d62f6ad0b3bd3756171"
server_key_iv: "83ea6dfa18007658c30f4b84047e7ead"
server_key_version: "encryptionkey@0"
I20260809 03:39:20.393337 698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "8f5ba61c0d384cc6b1c8c7d776b33230"
format_stamp: "Formatted at 2026-08-09 03:39:20 on dist-test-slave-jq4t"
server_key: "1088a89a35714d62f6ad0b3bd3756171"
server_key_iv: "83ea6dfa18007658c30f4b84047e7ead"
server_key_version: "encryptionkey@0"
I20260809 03:39:20.397544 698 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:39:20.400305 719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:20.401710 698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:39:20.401881 698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "8f5ba61c0d384cc6b1c8c7d776b33230"
format_stamp: "Formatted at 2026-08-09 03:39:20 on dist-test-slave-jq4t"
server_key: "1088a89a35714d62f6ad0b3bd3756171"
server_key_iv: "83ea6dfa18007658c30f4b84047e7ead"
server_key_version: "encryptionkey@0"
I20260809 03:39:20.402022 698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:20.426192 698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:20.430094 698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:20.430382 698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:20.439117 698 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32783
I20260809 03:39:20.439139 799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32783 every 8 connection(s)
I20260809 03:39:20.440263 698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:20.443426 801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:20.444440 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 698
I20260809 03:39:20.444614 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:20.444959 1921 external_mini_cluster.cc:1428] Setting key 3aa282b01f5b6748dc872111f95f4b5b
I20260809 03:39:20.450466 801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230: Bootstrap starting.
I20260809 03:39:20.453723 801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:20.454820 801 log.cc:824] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:39:20 dist-test-slave-jq4t krb5kdc[675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 03:39:20.457068 801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230: No bootstrap required, opened a new log
I20260809 03:39:20.460963 801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ba61c0d384cc6b1c8c7d776b33230" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32783 } }
I20260809 03:39:20.461163 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:20.446556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45231 (local address 127.0.0.1:32783)
0809 03:39:20.447056 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:20.447072 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:20.447112 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:39:20.447791 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:20.447810 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:20.448190 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:20.448595 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:20.449484 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:20.450548 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:20.451912 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:20.452580 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:20.452729 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:20.455900 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:20.455906 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:20.455923 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:20.455954 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:20.457890 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:20.458454 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:20.458460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:20.458466 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:20.458590 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:20.458959 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:20.458968 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:20.458970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:20.459405 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:20.459653 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:20.460004 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 03:39:20.460270 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":156,"threads_started":1}
I20260809 03:39:20.461253 801 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:20.461333 801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f5ba61c0d384cc6b1c8c7d776b33230, State: Initialized, Role: FOLLOWER
I20260809 03:39:20.461882 801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ba61c0d384cc6b1c8c7d776b33230" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32783 } }
I20260809 03:39:20.462044 801 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:20.462096 801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:20.462183 801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:20.463305 801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ba61c0d384cc6b1c8c7d776b33230" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32783 } }
I20260809 03:39:20.463692 801 leader_election.cc:302] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f5ba61c0d384cc6b1c8c7d776b33230; no voters:
I20260809 03:39:20.464128 801 leader_election.cc:288] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:20.464262 810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:20.464524 810 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [term 1 LEADER]: Becoming Leader. State: Replica: 8f5ba61c0d384cc6b1c8c7d776b33230, State: Running, Role: LEADER
I20260809 03:39:20.464857 810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ba61c0d384cc6b1c8c7d776b33230" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32783 } }
I20260809 03:39:20.464905 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:20.465618 801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:20.466682 817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f5ba61c0d384cc6b1c8c7d776b33230. Latest consensus state: current_term: 1 leader_uuid: "8f5ba61c0d384cc6b1c8c7d776b33230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ba61c0d384cc6b1c8c7d776b33230" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32783 } } }
I20260809 03:39:20.466830 817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:20.467209 811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f5ba61c0d384cc6b1c8c7d776b33230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5ba61c0d384cc6b1c8c7d776b33230" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32783 } } }
I20260809 03:39:20.467314 811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:20.467631 835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:20.471243 835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:20.478305 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:20.469033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35935 (local address 127.0.0.1:32783)
0809 03:39:20.469135 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:20.469140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:20.469314 (+ 174us) server_negotiation.cc:407] Connection header received
0809 03:39:20.469471 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:20.469474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:20.469532 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:20.469622 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:20.470524 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:20.471308 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:20.472428 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:20.472851 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:20.472936 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:20.474623 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:20.474627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:20.474630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:20.474657 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:20.476181 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:20.476811 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:20.476815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:20.476817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:20.476923 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:20.477353 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:20.477357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:20.477359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:20.477569 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:20.477751 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:20.478040 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 03:39:20.478143 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 03:39:20.479660 835 catalog_manager.cc:1347] Generated new cluster ID: d6d824eb085e40a18b6dfe655ff96677
I20260809 03:39:20.479722 835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:20.487671 835 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:20.488862 835 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:20.496856 835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f5ba61c0d384cc6b1c8c7d776b33230: Generated new TSK 0
I20260809 03:39:20.497695 835 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:20.505208 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 698
W20260809 03:39:20.527474 673 connection.cc:535] client connection to 127.0.0.1:32783 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32783): Connection reset by peer (error 104)
2026-08-09T03:39:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1676 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:20 dist-test-slave-jq4t krb5kdc[855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:20 dist-test-slave-jq4t krb5kdc[855](info): set up 2 sockets
Aug 09 03:39:20 dist-test-slave-jq4t krb5kdc[855](info): commencing operation
krb5kdc: starting...
W20260809 03:39:21.775949 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:21 dist-test-slave-jq4t krb5kdc[855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:39:21.912249 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:22.031239 875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:22.031584 875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:22.031630 875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:22.035887 875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:22.035975 875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:22.035995 875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:22.036013 875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:22.036029 875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:22.041798 875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43709
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:22.043114 875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:22.044263 875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:22.053807 887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:22.053843 885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:22.053825 884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:22.054250 875 server_base.cc:1034] running on GCE node
I20260809 03:39:22.054905 875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:22.056093 875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:22.057291 875 hybrid_clock.cc:648] HybridClock initialized: now 1786246762057272 us; error 36 us; skew 500 ppm
Aug 09 03:39:22 dist-test-slave-jq4t krb5kdc[855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246762, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:22.060185 875 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:39:22.061187 875 webserver.cc:460] Webserver started at http://127.0.0.1:45705/ using document root <none> and password file <none>
I20260809 03:39:22.061774 875 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:22.061873 875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:22.062122 875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:22.064307 875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "4cb9f5a2ad154e95b5bf3f43f2df49fa"
format_stamp: "Formatted at 2026-08-09 03:39:22 on dist-test-slave-jq4t"
server_key: "159a9b3b8f2faba235124703c6f71281"
server_key_iv: "954b3a4f719c5ab140c955e377764d73"
server_key_version: "encryptionkey@0"
I20260809 03:39:22.064996 875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "4cb9f5a2ad154e95b5bf3f43f2df49fa"
format_stamp: "Formatted at 2026-08-09 03:39:22 on dist-test-slave-jq4t"
server_key: "159a9b3b8f2faba235124703c6f71281"
server_key_iv: "954b3a4f719c5ab140c955e377764d73"
server_key_version: "encryptionkey@0"
I20260809 03:39:22.068769 875 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:39:22.071476 900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:22.072825 875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:22.072983 875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "4cb9f5a2ad154e95b5bf3f43f2df49fa"
format_stamp: "Formatted at 2026-08-09 03:39:22 on dist-test-slave-jq4t"
server_key: "159a9b3b8f2faba235124703c6f71281"
server_key_iv: "954b3a4f719c5ab140c955e377764d73"
server_key_version: "encryptionkey@0"
I20260809 03:39:22.073104 875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:22.083077 875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:22.086016 875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:22.086228 875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:22.093895 875 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43709
I20260809 03:39:22.093925 984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43709 every 8 connection(s)
I20260809 03:39:22.095001 875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:22.097945 985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:22.100663 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 875
I20260809 03:39:22.100826 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:22.101176 1921 external_mini_cluster.cc:1428] Setting key 3fb0b111a50581881f386d29ecdd38ab
I20260809 03:39:22.104364 985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa: Bootstrap starting.
I20260809 03:39:22.106614 985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:22.107365 985 log.cc:824] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:22.109611 985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa: No bootstrap required, opened a new log
I20260809 03:39:22.110802 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:22.102657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43883 (local address 127.0.0.1:43709)
0809 03:39:22.103099 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:22.103114 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:22.103150 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:39:22.103779 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:22.103795 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:22.104141 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:22.104465 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:22.105011 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:22.106042 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:22.107064 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:22.107602 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:22.107786 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:22.108773 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:22.108787 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:22.108806 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:22.109316 (+ 510us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:22.109390 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:22.109422 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:22.109574 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:22.109750 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 03:39:22.109885 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":146,"threads_started":1}
I20260809 03:39:22.113998 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:22.114084 985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb9f5a2ad154e95b5bf3f43f2df49fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43709 } }
I20260809 03:39:22.114348 985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:22.114449 985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb9f5a2ad154e95b5bf3f43f2df49fa, State: Initialized, Role: FOLLOWER
I20260809 03:39:22.114921 985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb9f5a2ad154e95b5bf3f43f2df49fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43709 } }
I20260809 03:39:22.115049 985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:22.115132 985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:22.115247 985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:22.116187 985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb9f5a2ad154e95b5bf3f43f2df49fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43709 } }
I20260809 03:39:22.116617 985 leader_election.cc:302] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cb9f5a2ad154e95b5bf3f43f2df49fa; no voters:
I20260809 03:39:22.116966 985 leader_election.cc:288] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:22.117031 1003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:22.117272 1003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [term 1 LEADER]: Becoming Leader. State: Replica: 4cb9f5a2ad154e95b5bf3f43f2df49fa, State: Running, Role: LEADER
I20260809 03:39:22.117583 1003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb9f5a2ad154e95b5bf3f43f2df49fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43709 } }
I20260809 03:39:22.118309 985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:22.118817 1006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cb9f5a2ad154e95b5bf3f43f2df49fa. Latest consensus state: current_term: 1 leader_uuid: "4cb9f5a2ad154e95b5bf3f43f2df49fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb9f5a2ad154e95b5bf3f43f2df49fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43709 } } }
I20260809 03:39:22.118949 1006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:22.119369 1005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cb9f5a2ad154e95b5bf3f43f2df49fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cb9f5a2ad154e95b5bf3f43f2df49fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43709 } } }
I20260809 03:39:22.119477 1005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:22.121650 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:22.116325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39283 (local address 127.0.0.1:43709)
0809 03:39:22.116437 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:22.116441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:22.116714 (+ 273us) server_negotiation.cc:407] Connection header received
0809 03:39:22.116872 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:22.116876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:22.116929 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:22.117064 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:22.117341 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:22.118013 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:22.119337 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:22.119673 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:22.119763 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:22.121083 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:22.121087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:22.121089 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:22.121104 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:22.121133 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:22.121147 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:22.121216 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:22.121479 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 03:39:22.121517 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 03:39:22.123234 1026 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:39:22.123317 1026 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:39:22.123412 1010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:22.124130 1010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:22.130913 1010 catalog_manager.cc:1347] Generated new cluster ID: b71917ddb6d74243980aaf42c2287660
I20260809 03:39:22.131021 1010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:22.136116 1010 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:22.137253 1010 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:22.142649 1010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cb9f5a2ad154e95b5bf3f43f2df49fa: Generated new TSK 0
I20260809 03:39:22.143383 1010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:22.150173 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 875
W20260809 03:39:22.170013 851 connection.cc:535] client connection to 127.0.0.1:43709 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43709): Connection reset by peer (error 104)
2026-08-09T03:39:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1642 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:22 dist-test-slave-jq4t krb5kdc[1035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:22 dist-test-slave-jq4t krb5kdc[1035](info): set up 2 sockets
Aug 09 03:39:22 dist-test-slave-jq4t krb5kdc[1035](info): commencing operation
krb5kdc: starting...
W20260809 03:39:23.413416 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:23 dist-test-slave-jq4t krb5kdc[1035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:39:23.547307 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35027
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:23.664523 1055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:23.664844 1055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:23.664924 1055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:23.668967 1055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:23.669068 1055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:23.669109 1055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:23.669162 1055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:23.669200 1055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:23.674628 1055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35027
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:23.675825 1055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:23.676997 1055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:23.686100 1062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:23.686100 1061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:23.686326 1055 server_base.cc:1034] running on GCE node
W20260809 03:39:23.686162 1064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:23.687045 1055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:23.688261 1055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:23.689426 1055 hybrid_clock.cc:648] HybridClock initialized: now 1786246763689412 us; error 63 us; skew 500 ppm
Aug 09 03:39:23 dist-test-slave-jq4t krb5kdc[1035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246763, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:23.692653 1055 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:39:23.693750 1055 webserver.cc:460] Webserver started at http://127.0.0.1:40291/ using document root <none> and password file <none>
I20260809 03:39:23.694425 1055 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:23.694557 1055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:23.694823 1055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:23.697270 1055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "22c04754cd0f481593c5b5ec222918d4"
format_stamp: "Formatted at 2026-08-09 03:39:23 on dist-test-slave-jq4t"
server_key: "55238897105e8238f2fa36aaa1b3a213"
server_key_iv: "801fd95cca6004b774e2af4882314dcd"
server_key_version: "encryptionkey@0"
I20260809 03:39:23.697907 1055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "22c04754cd0f481593c5b5ec222918d4"
format_stamp: "Formatted at 2026-08-09 03:39:23 on dist-test-slave-jq4t"
server_key: "55238897105e8238f2fa36aaa1b3a213"
server_key_iv: "801fd95cca6004b774e2af4882314dcd"
server_key_version: "encryptionkey@0"
I20260809 03:39:23.702165 1055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:39:23.704984 1077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:23.706295 1055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:23.706457 1055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "22c04754cd0f481593c5b5ec222918d4"
format_stamp: "Formatted at 2026-08-09 03:39:23 on dist-test-slave-jq4t"
server_key: "55238897105e8238f2fa36aaa1b3a213"
server_key_iv: "801fd95cca6004b774e2af4882314dcd"
server_key_version: "encryptionkey@0"
I20260809 03:39:23.706600 1055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:23.711737 1055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:23.715014 1055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:23.715246 1055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:23.723126 1055 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35027
I20260809 03:39:23.723137 1153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35027 every 8 connection(s)
I20260809 03:39:23.724201 1055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:23.725247 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 1055
I20260809 03:39:23.725394 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:23.725734 1921 external_mini_cluster.cc:1428] Setting key 7f09a2bd3a74a812d8d01c808b998839
I20260809 03:39:23.727878 1154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:23.731419 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:23.727268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45749 (local address 127.0.0.1:35027)
0809 03:39:23.727758 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:23.727773 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:23.727811 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:39:23.728605 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:23.728624 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:23.728920 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:23.729279 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:23.730150 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:23.730155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:23.730166 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:23.730177 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:23.730236 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:23.730255 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:23.730415 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:23.730487 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 03:39:23.730662 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":151,"threads_started":1}
I20260809 03:39:23.734716 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:23.734959 1154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4: Bootstrap starting.
I20260809 03:39:23.737807 1154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:23.738715 1154 log.cc:824] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:23.738960 1157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:23.737128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40757 (local address 127.0.0.1:35027)
0809 03:39:23.737275 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:23.737280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:23.737443 (+ 163us) server_negotiation.cc:407] Connection header received
0809 03:39:23.737558 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:23.737589 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:23.737641 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:23.737741 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:23.738551 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:23.738557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:23.738563 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:23.738580 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:23.738612 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:23.738626 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:23.738720 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:23.738798 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 03:39:23.738844 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:39:23.741150 1154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4: No bootstrap required, opened a new log
I20260809 03:39:23.745339 1154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22c04754cd0f481593c5b5ec222918d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35027 } }
I20260809 03:39:23.745609 1154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:23.745674 1154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22c04754cd0f481593c5b5ec222918d4, State: Initialized, Role: FOLLOWER
I20260809 03:39:23.746147 1154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22c04754cd0f481593c5b5ec222918d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35027 } }
I20260809 03:39:23.746307 1154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:23.746398 1154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:23.746523 1154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:23.747582 1154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22c04754cd0f481593c5b5ec222918d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35027 } }
I20260809 03:39:23.747965 1154 leader_election.cc:302] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22c04754cd0f481593c5b5ec222918d4; no voters:
I20260809 03:39:23.748353 1154 leader_election.cc:288] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:23.748543 1167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:23.748801 1167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [term 1 LEADER]: Becoming Leader. State: Replica: 22c04754cd0f481593c5b5ec222918d4, State: Running, Role: LEADER
I20260809 03:39:23.749177 1167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22c04754cd0f481593c5b5ec222918d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35027 } }
I20260809 03:39:23.749815 1154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:23.751058 1170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22c04754cd0f481593c5b5ec222918d4. Latest consensus state: current_term: 1 leader_uuid: "22c04754cd0f481593c5b5ec222918d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22c04754cd0f481593c5b5ec222918d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35027 } } }
I20260809 03:39:23.751199 1170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:23.751374 1168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22c04754cd0f481593c5b5ec222918d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22c04754cd0f481593c5b5ec222918d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35027 } } }
I20260809 03:39:23.751480 1168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:23.751768 1181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:23.754650 1181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:23.762391 1181 catalog_manager.cc:1347] Generated new cluster ID: 94a599ef34094621a41fc4dc60430a14
I20260809 03:39:23.762477 1181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:23.786521 1181 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:23.786885 1181 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:23.797392 1181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22c04754cd0f481593c5b5ec222918d4: Generated new TSK 0
I20260809 03:39:23.798319 1181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:23.804351 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 1055
2026-08-09T03:39:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1652 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:23 dist-test-slave-jq4t krb5kdc[1202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:23 dist-test-slave-jq4t krb5kdc[1202](info): set up 2 sockets
Aug 09 03:39:23 dist-test-slave-jq4t krb5kdc[1202](info): commencing operation
krb5kdc: starting...
W20260809 03:39:25.072400 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:25 dist-test-slave-jq4t krb5kdc[1202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:39:25.211346 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36145
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:25.330821 1224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:25.331192 1224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:25.331278 1224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:25.335484 1224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:25.335630 1224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:25.335668 1224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:25.335711 1224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:25.335767 1224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:25.341605 1224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:25.342909 1224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:25.344070 1224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:25.353330 1232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:25.353353 1231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:25.353477 1224 server_base.cc:1034] running on GCE node
W20260809 03:39:25.353663 1235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:25.354319 1224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:25.355543 1224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:25.356737 1224 hybrid_clock.cc:648] HybridClock initialized: now 1786246765356717 us; error 32 us; skew 500 ppm
Aug 09 03:39:25 dist-test-slave-jq4t krb5kdc[1202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246765, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:25.359673 1224 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:39:25.360708 1224 webserver.cc:460] Webserver started at http://127.0.0.1:45839/ using document root <none> and password file <none>
I20260809 03:39:25.361337 1224 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:25.361435 1224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:25.361690 1224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:25.363909 1224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "31c5d52ca0334f3cad49fff171fbf1f1"
format_stamp: "Formatted at 2026-08-09 03:39:25 on dist-test-slave-jq4t"
server_key: "0cca2758a80609342c736331336abb73"
server_key_iv: "6627a975c60cd38cd916ff2421e69123"
server_key_version: "encryptionkey@0"
I20260809 03:39:25.364567 1224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "31c5d52ca0334f3cad49fff171fbf1f1"
format_stamp: "Formatted at 2026-08-09 03:39:25 on dist-test-slave-jq4t"
server_key: "0cca2758a80609342c736331336abb73"
server_key_iv: "6627a975c60cd38cd916ff2421e69123"
server_key_version: "encryptionkey@0"
I20260809 03:39:25.368440 1224 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:39:25.371009 1247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:25.372231 1224 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:39:25.372387 1224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "31c5d52ca0334f3cad49fff171fbf1f1"
format_stamp: "Formatted at 2026-08-09 03:39:25 on dist-test-slave-jq4t"
server_key: "0cca2758a80609342c736331336abb73"
server_key_iv: "6627a975c60cd38cd916ff2421e69123"
server_key_version: "encryptionkey@0"
I20260809 03:39:25.372537 1224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:25.403142 1224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:25.406387 1224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:25.406620 1224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:25.414252 1224 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36145
I20260809 03:39:25.414289 1343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36145 every 8 connection(s)
I20260809 03:39:25.415333 1224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:25.418522 1345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:25.419804 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 1224
I20260809 03:39:25.419975 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:25.420291 1921 external_mini_cluster.cc:1428] Setting key 26e00d72822c231e0659491b19409159
I20260809 03:39:25.425235 1345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1: Bootstrap starting.
I20260809 03:39:25.427665 1345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:25.428750 1345 log.cc:824] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:39:25 dist-test-slave-jq4t krb5kdc[1202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 03:39:25.431357 1345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1: No bootstrap required, opened a new log
I20260809 03:39:25.435873 1349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:25.421906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33469 (local address 127.0.0.1:36145)
0809 03:39:25.422426 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:25.422443 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:25.422485 (+ 42us) server_negotiation.cc:407] Connection header received
0809 03:39:25.423194 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:25.423213 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:25.423578 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:25.423956 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:25.424909 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:25.425981 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:25.426891 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:25.427390 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:25.427569 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:25.430724 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:25.430730 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:25.430746 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:25.430774 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:25.432740 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:25.433291 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:25.433296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:25.433301 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:25.433418 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:25.433750 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:25.433753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:25.433754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:25.434209 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:25.434367 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:25.434700 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 03:39:25.435055 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":167,"threads_started":1}
I20260809 03:39:25.436069 1345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31c5d52ca0334f3cad49fff171fbf1f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } }
I20260809 03:39:25.436378 1345 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:25.436462 1345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31c5d52ca0334f3cad49fff171fbf1f1, State: Initialized, Role: FOLLOWER
I20260809 03:39:25.436993 1345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31c5d52ca0334f3cad49fff171fbf1f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } }
I20260809 03:39:25.437165 1345 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:25.437255 1345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:25.437383 1345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:25.438405 1345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31c5d52ca0334f3cad49fff171fbf1f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } }
I20260809 03:39:25.438822 1345 leader_election.cc:302] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31c5d52ca0334f3cad49fff171fbf1f1; no voters:
I20260809 03:39:25.439235 1345 leader_election.cc:288] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:25.439352 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:25.439339 1355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:25.439572 1355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [term 1 LEADER]: Becoming Leader. State: Replica: 31c5d52ca0334f3cad49fff171fbf1f1, State: Running, Role: LEADER
I20260809 03:39:25.439872 1355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31c5d52ca0334f3cad49fff171fbf1f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } }
I20260809 03:39:25.440762 1345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:25.441715 1356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31c5d52ca0334f3cad49fff171fbf1f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31c5d52ca0334f3cad49fff171fbf1f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } } }
I20260809 03:39:25.441725 1357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31c5d52ca0334f3cad49fff171fbf1f1. Latest consensus state: current_term: 1 leader_uuid: "31c5d52ca0334f3cad49fff171fbf1f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31c5d52ca0334f3cad49fff171fbf1f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } } }
I20260809 03:39:25.441855 1356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:25.441883 1357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:25.442256 1375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:25.445966 1375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:25.452837 1375 catalog_manager.cc:1347] Generated new cluster ID: c675adef6be944d8b3cf9a0d1062ca43
I20260809 03:39:25.453020 1375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:25.453730 1349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:25.443668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39131 (local address 127.0.0.1:36145)
0809 03:39:25.443794 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:25.443798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:25.444018 (+ 220us) server_negotiation.cc:407] Connection header received
0809 03:39:25.444235 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:25.444239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:25.444291 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:25.444405 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:25.445428 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:25.446211 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:25.447675 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:25.448022 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:25.448166 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:25.449686 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:25.449690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:25.449692 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:25.449721 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:25.451641 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:25.452124 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:25.452129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:25.452131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:25.452225 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:25.452589 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:25.452593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:25.452595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:25.452877 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:25.453013 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:25.453406 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 03:39:25.453559 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:39:25.470331 1375 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:25.471665 1375 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:25.479650 1375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31c5d52ca0334f3cad49fff171fbf1f1: Generated new TSK 0
I20260809 03:39:25.480429 1375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:25.488363 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 1224
2026-08-09T03:39:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1683 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:25 dist-test-slave-jq4t krb5kdc[1394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:25 dist-test-slave-jq4t krb5kdc[1394](info): set up 2 sockets
Aug 09 03:39:25 dist-test-slave-jq4t krb5kdc[1394](info): commencing operation
krb5kdc: starting...
W20260809 03:39:26.753023 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:26 dist-test-slave-jq4t krb5kdc[1394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:39:26.888067 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:27.011165 1416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:27.011523 1416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:27.011631 1416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:27.015873 1416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:27.016007 1416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:27.016048 1416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:27.016088 1416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:27.016121 1416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:27.021943 1416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:27.023218 1416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:27.024370 1416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:27.033442 1424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:27.033412 1416 server_base.cc:1034] running on GCE node
W20260809 03:39:27.033419 1421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:27.033377 1426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:27.034330 1416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:27.035532 1416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:27.036724 1416 hybrid_clock.cc:648] HybridClock initialized: now 1786246767036709 us; error 39 us; skew 500 ppm
Aug 09 03:39:27 dist-test-slave-jq4t krb5kdc[1394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246767, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:27.039742 1416 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:39:27.040819 1416 webserver.cc:460] Webserver started at http://127.0.0.1:42851/ using document root <none> and password file <none>
I20260809 03:39:27.041396 1416 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:27.041462 1416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:27.041646 1416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:27.043867 1416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "cf2442aeecbc426986df2b05d216f8b0"
format_stamp: "Formatted at 2026-08-09 03:39:27 on dist-test-slave-jq4t"
server_key: "0d7fc66f48a27e172bdbc18d198420fd"
server_key_iv: "b9d2b7ecd9746d1ebcb4213be3184c83"
server_key_version: "encryptionkey@0"
I20260809 03:39:27.044453 1416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "cf2442aeecbc426986df2b05d216f8b0"
format_stamp: "Formatted at 2026-08-09 03:39:27 on dist-test-slave-jq4t"
server_key: "0d7fc66f48a27e172bdbc18d198420fd"
server_key_iv: "b9d2b7ecd9746d1ebcb4213be3184c83"
server_key_version: "encryptionkey@0"
I20260809 03:39:27.049281 1416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:39:27.052129 1441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:27.053679 1416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:39:27.053845 1416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "cf2442aeecbc426986df2b05d216f8b0"
format_stamp: "Formatted at 2026-08-09 03:39:27 on dist-test-slave-jq4t"
server_key: "0d7fc66f48a27e172bdbc18d198420fd"
server_key_iv: "b9d2b7ecd9746d1ebcb4213be3184c83"
server_key_version: "encryptionkey@0"
I20260809 03:39:27.053974 1416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:27.075419 1416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:27.079103 1416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:27.079281 1416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:27.086894 1416 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46225
I20260809 03:39:27.086959 1531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46225 every 8 connection(s)
I20260809 03:39:27.087970 1416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:27.091064 1533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:27.096807 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 1416
I20260809 03:39:27.096975 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:27.097239 1921 external_mini_cluster.cc:1428] Setting key 2755ec456288543d01f1eba733ae0ad7
I20260809 03:39:27.097435 1533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0: Bootstrap starting.
I20260809 03:39:27.100153 1533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:27.101166 1533 log.cc:824] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:27.103585 1533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0: No bootstrap required, opened a new log
I20260809 03:39:27.107100 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:27.098777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40125 (local address 127.0.0.1:46225)
0809 03:39:27.099292 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:27.099310 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:27.099354 (+ 44us) server_negotiation.cc:407] Connection header received
0809 03:39:27.100030 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:27.100047 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:27.100473 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:27.100834 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:27.101259 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:27.102540 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:27.103827 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:27.104316 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:27.104441 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:27.105355 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:27.105361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:27.105375 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:27.105772 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:27.105828 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:27.105851 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:27.105944 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:27.106103 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 03:39:27.106342 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":181,"threads_started":1}
I20260809 03:39:27.107903 1533 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2442aeecbc426986df2b05d216f8b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46225 } }
I20260809 03:39:27.108158 1533 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:27.108233 1533 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf2442aeecbc426986df2b05d216f8b0, State: Initialized, Role: FOLLOWER
I20260809 03:39:27.108767 1533 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2442aeecbc426986df2b05d216f8b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46225 } }
I20260809 03:39:27.108928 1533 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:27.109025 1533 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:27.109140 1533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:27.110160 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:27.110193 1533 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2442aeecbc426986df2b05d216f8b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46225 } }
I20260809 03:39:27.110565 1533 leader_election.cc:302] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf2442aeecbc426986df2b05d216f8b0; no voters:
I20260809 03:39:27.110975 1533 leader_election.cc:288] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:27.111088 1542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:27.111352 1542 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [term 1 LEADER]: Becoming Leader. State: Replica: cf2442aeecbc426986df2b05d216f8b0, State: Running, Role: LEADER
I20260809 03:39:27.111726 1542 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2442aeecbc426986df2b05d216f8b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46225 } }
I20260809 03:39:27.112416 1533 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:27.113229 1545 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf2442aeecbc426986df2b05d216f8b0. Latest consensus state: current_term: 1 leader_uuid: "cf2442aeecbc426986df2b05d216f8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2442aeecbc426986df2b05d216f8b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46225 } } }
I20260809 03:39:27.113453 1545 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:27.113474 1543 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf2442aeecbc426986df2b05d216f8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2442aeecbc426986df2b05d216f8b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46225 } } }
I20260809 03:39:27.113590 1543 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:27.113924 1563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:27.117722 1563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:27.118203 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:27.112568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36981 (local address 127.0.0.1:46225)
0809 03:39:27.112659 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:27.112664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:27.113690 (+ 1026us) server_negotiation.cc:407] Connection header received
0809 03:39:27.113747 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:27.113750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:27.113804 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:27.113886 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:27.114388 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:27.115246 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:27.116457 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:27.116723 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:27.116841 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:27.117777 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:27.117782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:27.117785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:27.117799 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:27.117830 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:27.117843 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:27.117911 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:27.118034 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 03:39:27.118093 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 03:39:27.124014 1563 catalog_manager.cc:1347] Generated new cluster ID: 350aca5f029a47c7bf919c51117bf1f5
I20260809 03:39:27.124099 1563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:27.145465 1563 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:27.146617 1563 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:27.154232 1563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf2442aeecbc426986df2b05d216f8b0: Generated new TSK 0
I20260809 03:39:27.155084 1563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:27.162322 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 1416
2026-08-09T03:39:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1669 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:27 dist-test-slave-jq4t krb5kdc[1577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:27 dist-test-slave-jq4t krb5kdc[1577](info): set up 2 sockets
Aug 09 03:39:27 dist-test-slave-jq4t krb5kdc[1577](info): commencing operation
krb5kdc: starting...
W20260809 03:39:28.422947 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:28 dist-test-slave-jq4t krb5kdc[1577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:39:28.560303 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36951
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:28.680342 1598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:28.680716 1598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:28.680766 1598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:28.684798 1598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:28.684862 1598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:28.684882 1598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:28.684898 1598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:28.684913 1598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:28.690307 1598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:28.691459 1598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:28.692471 1598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:28.701835 1605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:28.701797 1611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:28.701872 1604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:28.702150 1598 server_base.cc:1034] running on GCE node
I20260809 03:39:28.702872 1598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:28.704066 1598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:28.705269 1598 hybrid_clock.cc:648] HybridClock initialized: now 1786246768705241 us; error 46 us; skew 500 ppm
Aug 09 03:39:28 dist-test-slave-jq4t krb5kdc[1577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246768, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:28.708181 1598 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:39:28.709177 1598 webserver.cc:460] Webserver started at http://127.0.0.1:36549/ using document root <none> and password file <none>
I20260809 03:39:28.709791 1598 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:28.709892 1598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:28.710143 1598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:28.712316 1598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "8563a7d4f15a43f2adf9c55d4d95827a"
format_stamp: "Formatted at 2026-08-09 03:39:28 on dist-test-slave-jq4t"
server_key: "e369d0ddf7b9d176b75d225afed0b9f2"
server_key_iv: "14217afdab6e1f09a25d325ad658cb63"
server_key_version: "encryptionkey@0"
I20260809 03:39:28.712996 1598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "8563a7d4f15a43f2adf9c55d4d95827a"
format_stamp: "Formatted at 2026-08-09 03:39:28 on dist-test-slave-jq4t"
server_key: "e369d0ddf7b9d176b75d225afed0b9f2"
server_key_iv: "14217afdab6e1f09a25d325ad658cb63"
server_key_version: "encryptionkey@0"
I20260809 03:39:28.716965 1598 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:39:28.719563 1621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:28.720919 1598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:28.721072 1598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "8563a7d4f15a43f2adf9c55d4d95827a"
format_stamp: "Formatted at 2026-08-09 03:39:28 on dist-test-slave-jq4t"
server_key: "e369d0ddf7b9d176b75d225afed0b9f2"
server_key_iv: "14217afdab6e1f09a25d325ad658cb63"
server_key_version: "encryptionkey@0"
I20260809 03:39:28.721203 1598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:28.726142 1598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:28.729635 1598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:28.729849 1598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:28.737649 1598 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36951
I20260809 03:39:28.737730 1708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36951 every 8 connection(s)
I20260809 03:39:28.738749 1598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:28.741899 1710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:28.747756 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 1598
I20260809 03:39:28.747924 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:28.748224 1921 external_mini_cluster.cc:1428] Setting key c943faf7dd93fb5c9d770870d4fa93d8
I20260809 03:39:28.748576 1710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a: Bootstrap starting.
I20260809 03:39:28.751202 1710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:28.752206 1710 log.cc:824] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:28.753779 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:28.749689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40709 (local address 127.0.0.1:36951)
0809 03:39:28.750135 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:28.750152 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:28.750189 (+ 37us) server_negotiation.cc:407] Connection header received
0809 03:39:28.750877 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:28.750897 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:28.751195 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:28.751541 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:28.752468 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:28.752506 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:28.752521 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:28.752538 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:28.752599 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:28.752621 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:28.752752 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:28.752898 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 03:39:28.753079 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":138,"threads_started":1}
I20260809 03:39:28.754423 1710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a: No bootstrap required, opened a new log
I20260809 03:39:28.756580 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:28.758404 1710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8563a7d4f15a43f2adf9c55d4d95827a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36951 } }
I20260809 03:39:28.758709 1710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:28.758823 1710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8563a7d4f15a43f2adf9c55d4d95827a, State: Initialized, Role: FOLLOWER
I20260809 03:39:28.759370 1710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8563a7d4f15a43f2adf9c55d4d95827a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36951 } }
I20260809 03:39:28.759578 1710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:28.759660 1710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:28.759786 1710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:28.760525 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:28.758671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33503 (local address 127.0.0.1:36951)
0809 03:39:28.758790 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:28.758795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:28.759031 (+ 236us) server_negotiation.cc:407] Connection header received
0809 03:39:28.759174 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:28.759177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:28.759217 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:28.759284 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:28.760095 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:28.760099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:28.760101 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:28.760113 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:28.760140 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:28.760153 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:28.760249 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:28.760328 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 03:39:28.760368 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:39:28.761006 1710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8563a7d4f15a43f2adf9c55d4d95827a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36951 } }
I20260809 03:39:28.761408 1710 leader_election.cc:302] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8563a7d4f15a43f2adf9c55d4d95827a; no voters:
I20260809 03:39:28.761745 1710 leader_election.cc:288] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:28.761862 1723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:28.762118 1723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [term 1 LEADER]: Becoming Leader. State: Replica: 8563a7d4f15a43f2adf9c55d4d95827a, State: Running, Role: LEADER
I20260809 03:39:28.762475 1723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8563a7d4f15a43f2adf9c55d4d95827a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36951 } }
I20260809 03:39:28.763023 1710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:28.764200 1725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8563a7d4f15a43f2adf9c55d4d95827a. Latest consensus state: current_term: 1 leader_uuid: "8563a7d4f15a43f2adf9c55d4d95827a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8563a7d4f15a43f2adf9c55d4d95827a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36951 } } }
I20260809 03:39:28.764345 1725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:28.764761 1735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:28.764731 1724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8563a7d4f15a43f2adf9c55d4d95827a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8563a7d4f15a43f2adf9c55d4d95827a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36951 } } }
I20260809 03:39:28.764897 1724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:28.768178 1735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:28.773767 1735 catalog_manager.cc:1347] Generated new cluster ID: 56fcce3682ff46629a711b2f85c797e6
I20260809 03:39:28.773849 1735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:28.786275 1735 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:28.786686 1735 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:28.791061 1735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8563a7d4f15a43f2adf9c55d4d95827a: Generated new TSK 0
I20260809 03:39:28.791749 1735 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:28.796093 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 1598
2026-08-09T03:39:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1631 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:28 dist-test-slave-jq4t krb5kdc[1758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:28 dist-test-slave-jq4t krb5kdc[1758](info): set up 2 sockets
Aug 09 03:39:28 dist-test-slave-jq4t krb5kdc[1758](info): commencing operation
krb5kdc: starting...
W20260809 03:39:30.045722 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:30 dist-test-slave-jq4t krb5kdc[1758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:39:30.180280 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33453
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:30.301043 1775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:30.301393 1775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:30.301473 1775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:30.305625 1775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:30.305743 1775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:30.305780 1775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:30.305822 1775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:30.305858 1775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:30.311329 1775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:30.312505 1775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:30.313593 1775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:30.322460 1782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:30.322445 1791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:30.322533 1775 server_base.cc:1034] running on GCE node
W20260809 03:39:30.322458 1785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:30.323436 1775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:30.324672 1775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:30.325848 1775 hybrid_clock.cc:648] HybridClock initialized: now 1786246770325825 us; error 33 us; skew 500 ppm
Aug 09 03:39:30 dist-test-slave-jq4t krb5kdc[1758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246770, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:30.328778 1775 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:39:30.329787 1775 webserver.cc:460] Webserver started at http://127.0.0.1:42539/ using document root <none> and password file <none>
I20260809 03:39:30.330399 1775 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:30.330497 1775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:30.330744 1775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:30.332973 1775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "bc33fdee362f4521829923f143aaefd2"
format_stamp: "Formatted at 2026-08-09 03:39:30 on dist-test-slave-jq4t"
server_key: "cb378502652b500238ccaa23316c9ecc"
server_key_iv: "6684dd77856fb3699ed6440e47b1d23f"
server_key_version: "encryptionkey@0"
I20260809 03:39:30.333581 1775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "bc33fdee362f4521829923f143aaefd2"
format_stamp: "Formatted at 2026-08-09 03:39:30 on dist-test-slave-jq4t"
server_key: "cb378502652b500238ccaa23316c9ecc"
server_key_iv: "6684dd77856fb3699ed6440e47b1d23f"
server_key_version: "encryptionkey@0"
I20260809 03:39:30.337786 1775 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 03:39:30.340380 1809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:30.341686 1775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:39:30.341841 1775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "bc33fdee362f4521829923f143aaefd2"
format_stamp: "Formatted at 2026-08-09 03:39:30 on dist-test-slave-jq4t"
server_key: "cb378502652b500238ccaa23316c9ecc"
server_key_iv: "6684dd77856fb3699ed6440e47b1d23f"
server_key_version: "encryptionkey@0"
I20260809 03:39:30.341971 1775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:30.368675 1775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:30.371977 1775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:30.372221 1775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:30.380874 1775 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33453
I20260809 03:39:30.380956 1887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33453 every 8 connection(s)
I20260809 03:39:30.382061 1775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:30.385417 1889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:30.388624 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 1775
I20260809 03:39:30.388788 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:30.389066 1921 external_mini_cluster.cc:1428] Setting key e11daf284f017a2812e680091b46b4e6
I20260809 03:39:30.392280 1889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2: Bootstrap starting.
I20260809 03:39:30.394742 1889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:30.395567 1889 log.cc:824] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:30.397909 1889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2: No bootstrap required, opened a new log
Aug 09 03:39:30 dist-test-slave-jq4t krb5kdc[1758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 03:39:30.401844 1889 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc33fdee362f4521829923f143aaefd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33453 } }
I20260809 03:39:30.402122 1889 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:30.402210 1889 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc33fdee362f4521829923f143aaefd2, State: Initialized, Role: FOLLOWER
I20260809 03:39:30.402745 1889 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc33fdee362f4521829923f143aaefd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33453 } }
I20260809 03:39:30.402907 1889 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:30.402998 1889 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:30.403105 1889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:30.404086 1889 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc33fdee362f4521829923f143aaefd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33453 } }
I20260809 03:39:30.404531 1889 leader_election.cc:302] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc33fdee362f4521829923f143aaefd2; no voters:
I20260809 03:39:30.404946 1889 leader_election.cc:288] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:30.405079 1897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:30.405308 1897 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [term 1 LEADER]: Becoming Leader. State: Replica: bc33fdee362f4521829923f143aaefd2, State: Running, Role: LEADER
I20260809 03:39:30.405745 1897 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc33fdee362f4521829923f143aaefd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33453 } }
I20260809 03:39:30.406462 1889 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:30.409535 1898 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc33fdee362f4521829923f143aaefd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc33fdee362f4521829923f143aaefd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33453 } } }
I20260809 03:39:30.409834 1898 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:30.409592 1900 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc33fdee362f4521829923f143aaefd2. Latest consensus state: current_term: 1 leader_uuid: "bc33fdee362f4521829923f143aaefd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc33fdee362f4521829923f143aaefd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33453 } } }
I20260809 03:39:30.410089 1900 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:30.409765 1893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:30.390817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:43607 (local address 127.0.0.1:33453)
0809 03:39:30.391362 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:30.391381 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:30.391418 (+ 37us) server_negotiation.cc:407] Connection header received
0809 03:39:30.392090 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:30.392111 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:30.392579 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:30.392950 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:30.393838 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:30.394900 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:30.395876 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:30.396392 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:30.396612 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:30.399656 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:30.399661 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:30.399677 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:30.399707 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:30.405646 (+ 5939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:30.406373 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:30.406380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:30.406387 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:30.406539 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:30.406943 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:30.406947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:30.406950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:30.407513 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:30.407743 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:30.408245 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 03:39:30.408558 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":163,"threads_started":1}
W20260809 03:39:30.412676 1918 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:39:30.412765 1918 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:39:30.412842 1928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:30.413560 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:30.413823 1928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:30.420444 1928 catalog_manager.cc:1347] Generated new cluster ID: f696605321004d88b2af5f03a3f19aa6
I20260809 03:39:30.420676 1928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:30.427646 1893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:30.415663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:51473 (local address 127.0.0.1:33453)
0809 03:39:30.415785 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:30.415789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:30.418304 (+ 2515us) server_negotiation.cc:407] Connection header received
0809 03:39:30.418454 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:30.418457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:30.418515 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:30.418608 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:30.419564 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:30.420346 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:30.421500 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:30.421871 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:30.421992 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:30.423724 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:30.423729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:30.423732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:30.423760 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:30.425605 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:30.426160 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:30.426164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:30.426166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:30.426264 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:30.426664 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:30.426667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:30.426669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:30.426909 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:30.427118 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:30.427356 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 03:39:30.427466 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":45}
I20260809 03:39:30.434111 1928 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:30.435470 1928 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:30.447824 1928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc33fdee362f4521829923f143aaefd2: Generated new TSK 0
I20260809 03:39:30.448734 1928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:30.459573 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 1775
W20260809 03:39:30.474655 1755 connection.cc:535] client connection to 127.0.0.1:33453 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33453): Connection reset by peer (error 104)
2026-08-09T03:39:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1667 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:30 dist-test-slave-jq4t krb5kdc[1943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:30 dist-test-slave-jq4t krb5kdc[1943](info): set up 2 sockets
Aug 09 03:39:30 dist-test-slave-jq4t krb5kdc[1943](info): commencing operation
krb5kdc: starting...
W20260809 03:39:31.731552 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:31 dist-test-slave-jq4t krb5kdc[1943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:39:31.867904 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39449
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:31.992017 1965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:31.992321 1965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:31.992367 1965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:31.996507 1965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:31.996575 1965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:31.996595 1965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:31.996613 1965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:31.996628 1965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:32.002123 1965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:32.003409 1965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:32.004526 1965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:32.013423 1972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:32.013423 1976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:32.013504 1974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:32.013777 1965 server_base.cc:1034] running on GCE node
I20260809 03:39:32.014447 1965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:32.015636 1965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:32.016835 1965 hybrid_clock.cc:648] HybridClock initialized: now 1786246772016810 us; error 53 us; skew 500 ppm
Aug 09 03:39:32 dist-test-slave-jq4t krb5kdc[1943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246772, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:32.019907 1965 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:39:32.020984 1965 webserver.cc:460] Webserver started at http://127.0.0.1:41161/ using document root <none> and password file <none>
I20260809 03:39:32.021595 1965 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:32.021689 1965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:32.021947 1965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:32.024111 1965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "2199072e80024ba988db61eb8ec00e5c"
format_stamp: "Formatted at 2026-08-09 03:39:32 on dist-test-slave-jq4t"
server_key: "c98324af9b440a17c365401633665b77"
server_key_iv: "19a3b7c6cda1173cd6eb8f5888567527"
server_key_version: "encryptionkey@0"
I20260809 03:39:32.024749 1965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "2199072e80024ba988db61eb8ec00e5c"
format_stamp: "Formatted at 2026-08-09 03:39:32 on dist-test-slave-jq4t"
server_key: "c98324af9b440a17c365401633665b77"
server_key_iv: "19a3b7c6cda1173cd6eb8f5888567527"
server_key_version: "encryptionkey@0"
I20260809 03:39:32.028657 1965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:39:32.031194 1985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:32.032420 1965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:39:32.032593 1965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "2199072e80024ba988db61eb8ec00e5c"
format_stamp: "Formatted at 2026-08-09 03:39:32 on dist-test-slave-jq4t"
server_key: "c98324af9b440a17c365401633665b77"
server_key_iv: "19a3b7c6cda1173cd6eb8f5888567527"
server_key_version: "encryptionkey@0"
I20260809 03:39:32.032718 1965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:32.058058 1965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:32.061674 1965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:32.061910 1965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:32.069905 1965 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39449
I20260809 03:39:32.069923 2070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39449 every 8 connection(s)
I20260809 03:39:32.070955 1965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:32.073972 2071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:32.076412 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 1965
I20260809 03:39:32.076591 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:32.076918 1921 external_mini_cluster.cc:1428] Setting key e3a90e85b16e203de94f6a3c194c715d
I20260809 03:39:32.080861 2071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c: Bootstrap starting.
I20260809 03:39:32.083314 2071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:32.084113 2071 log.cc:824] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:32.086716 2071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c: No bootstrap required, opened a new log
I20260809 03:39:32.087015 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:32.078469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:40227 (local address 127.0.0.1:39449)
0809 03:39:32.078926 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:32.078942 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:32.078978 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:39:32.079783 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:32.079804 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:32.080176 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:32.080567 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:32.081036 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:32.082053 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:32.082991 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:32.083543 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:32.083715 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:32.084815 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:32.084820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:32.084835 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:32.085378 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:32.085452 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:32.085485 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:32.085641 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:32.085836 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 03:39:32.085986 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":148,"threads_started":1}
I20260809 03:39:32.090685 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:32.091459 2071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2199072e80024ba988db61eb8ec00e5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39449 } }
I20260809 03:39:32.091841 2071 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:32.091957 2071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2199072e80024ba988db61eb8ec00e5c, State: Initialized, Role: FOLLOWER
I20260809 03:39:32.092582 2071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2199072e80024ba988db61eb8ec00e5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39449 } }
I20260809 03:39:32.092778 2071 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:32.092849 2071 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:32.092968 2071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:32.094209 2071 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2199072e80024ba988db61eb8ec00e5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39449 } }
I20260809 03:39:32.094635 2071 leader_election.cc:302] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2199072e80024ba988db61eb8ec00e5c; no voters:
I20260809 03:39:32.095003 2071 leader_election.cc:288] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:32.095108 2083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:32.095383 2083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [term 1 LEADER]: Becoming Leader. State: Replica: 2199072e80024ba988db61eb8ec00e5c, State: Running, Role: LEADER
I20260809 03:39:32.095742 2083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2199072e80024ba988db61eb8ec00e5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39449 } }
I20260809 03:39:32.096760 2071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:32.097186 2083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2199072e80024ba988db61eb8ec00e5c. Latest consensus state: current_term: 1 leader_uuid: "2199072e80024ba988db61eb8ec00e5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2199072e80024ba988db61eb8ec00e5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39449 } } }
I20260809 03:39:32.097333 2083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:32.097981 2085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2199072e80024ba988db61eb8ec00e5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2199072e80024ba988db61eb8ec00e5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39449 } } }
I20260809 03:39:32.098331 2085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:32.098804 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:32.093340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:40751 (local address 127.0.0.1:39449)
0809 03:39:32.093491 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:32.093496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:32.093702 (+ 206us) server_negotiation.cc:407] Connection header received
0809 03:39:32.093886 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:32.093889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:32.093949 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:32.094032 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:32.094329 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:32.095062 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:32.096302 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:32.096710 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:32.096825 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:32.097960 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:32.097966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:32.097971 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:32.098000 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:32.098039 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:32.098059 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:32.098169 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:32.098557 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 03:39:32.098599 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:39:32.098688 2098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:32.103232 2098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:32.109118 2098 catalog_manager.cc:1347] Generated new cluster ID: e9134415861e4b5dbfa0a58699fc4b31
I20260809 03:39:32.109196 2098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:32.126176 2098 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:32.127255 2098 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:32.144104 2098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2199072e80024ba988db61eb8ec00e5c: Generated new TSK 0
I20260809 03:39:32.145155 2098 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:32.155663 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 1965
W20260809 03:39:32.169521 1940 connection.cc:535] client connection to 127.0.0.1:39449 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39449): Connection reset by peer (error 104)
2026-08-09T03:39:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1694 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:32 dist-test-slave-jq4t krb5kdc[2117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:32 dist-test-slave-jq4t krb5kdc[2117](info): set up 2 sockets
Aug 09 03:39:32 dist-test-slave-jq4t krb5kdc[2117](info): commencing operation
krb5kdc: starting...
W20260809 03:39:33.415495 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:33 dist-test-slave-jq4t krb5kdc[2117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:39:33.551987 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39407
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:33.671921 2139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:33.672286 2139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:33.672384 2139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:33.676676 2139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:33.676782 2139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:33.676816 2139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:33.676854 2139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:33.676890 2139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:33.682642 2139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:33.683925 2139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:33.685241 2139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:33.694139 2155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:33.694164 2163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:33.694164 2152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:33.694478 2139 server_base.cc:1034] running on GCE node
I20260809 03:39:33.695084 2139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:33.696182 2139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:33.697356 2139 hybrid_clock.cc:648] HybridClock initialized: now 1786246773697330 us; error 37 us; skew 500 ppm
Aug 09 03:39:33 dist-test-slave-jq4t krb5kdc[2117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:33.703496 2139 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:39:33.704648 2139 webserver.cc:460] Webserver started at http://127.0.0.1:34049/ using document root <none> and password file <none>
I20260809 03:39:33.705292 2139 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:33.705394 2139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:33.705653 2139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:33.707980 2139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "298f874addaa45c1a615420b090720d4"
format_stamp: "Formatted at 2026-08-09 03:39:33 on dist-test-slave-jq4t"
server_key: "19df09b9cdfa732a430a481449c76fb5"
server_key_iv: "5e4f6bd60a94cc70d1aad448382c41ce"
server_key_version: "encryptionkey@0"
I20260809 03:39:33.708657 2139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "298f874addaa45c1a615420b090720d4"
format_stamp: "Formatted at 2026-08-09 03:39:33 on dist-test-slave-jq4t"
server_key: "19df09b9cdfa732a430a481449c76fb5"
server_key_iv: "5e4f6bd60a94cc70d1aad448382c41ce"
server_key_version: "encryptionkey@0"
I20260809 03:39:33.712848 2139 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:39:33.715478 2173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:33.716842 2139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:33.717028 2139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "298f874addaa45c1a615420b090720d4"
format_stamp: "Formatted at 2026-08-09 03:39:33 on dist-test-slave-jq4t"
server_key: "19df09b9cdfa732a430a481449c76fb5"
server_key_iv: "5e4f6bd60a94cc70d1aad448382c41ce"
server_key_version: "encryptionkey@0"
I20260809 03:39:33.717156 2139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:33.722064 2139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:33.725281 2139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:33.725489 2139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:33.732944 2139 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39407
I20260809 03:39:33.732971 2267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39407 every 8 connection(s)
I20260809 03:39:33.733884 2139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:33.736841 2268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:33.740370 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 2139
I20260809 03:39:33.740572 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:33.740893 1921 external_mini_cluster.cc:1428] Setting key 33f52393e7d059006920623e63ed459f
I20260809 03:39:33.743568 2268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4: Bootstrap starting.
I20260809 03:39:33.746217 2268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:33.746522 2273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:33.742424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:41221 (local address 127.0.0.1:39407)
0809 03:39:33.742938 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:33.742954 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:33.742993 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:39:33.743637 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:33.743655 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:33.744005 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:33.744353 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:33.745245 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:33.745250 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:33.745262 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:33.745272 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:33.745328 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:33.745348 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:33.745514 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:33.745589 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 03:39:33.745761 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":153,"threads_started":1}
I20260809 03:39:33.747184 2268 log.cc:824] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:33.749555 2268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4: No bootstrap required, opened a new log
I20260809 03:39:33.749738 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:33.753647 2273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:33.751822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:54247 (local address 127.0.0.1:39407)
0809 03:39:33.751964 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:33.751969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:33.752178 (+ 209us) server_negotiation.cc:407] Connection header received
0809 03:39:33.752286 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:33.752289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:33.752321 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:33.752390 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:33.753227 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:33.753230 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:33.753232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:33.753243 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:33.753269 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:33.753279 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:33.753351 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:33.753464 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 03:39:33.753499 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:39:33.754238 2268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298f874addaa45c1a615420b090720d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39407 } }
I20260809 03:39:33.754511 2268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:33.754611 2268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 298f874addaa45c1a615420b090720d4, State: Initialized, Role: FOLLOWER
I20260809 03:39:33.755126 2268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298f874addaa45c1a615420b090720d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39407 } }
I20260809 03:39:33.755292 2268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:33.755363 2268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:33.755466 2268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:33.756556 2268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298f874addaa45c1a615420b090720d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39407 } }
I20260809 03:39:33.756953 2268 leader_election.cc:302] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 298f874addaa45c1a615420b090720d4; no voters:
I20260809 03:39:33.757364 2268 leader_election.cc:288] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:33.757539 2286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:33.757833 2286 raft_consensus.cc:686] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [term 1 LEADER]: Becoming Leader. State: Replica: 298f874addaa45c1a615420b090720d4, State: Running, Role: LEADER
I20260809 03:39:33.758185 2286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298f874addaa45c1a615420b090720d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39407 } }
I20260809 03:39:33.758881 2268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:33.760167 2289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 298f874addaa45c1a615420b090720d4. Latest consensus state: current_term: 1 leader_uuid: "298f874addaa45c1a615420b090720d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298f874addaa45c1a615420b090720d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39407 } } }
I20260809 03:39:33.760313 2289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:33.760637 2287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "298f874addaa45c1a615420b090720d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298f874addaa45c1a615420b090720d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39407 } } }
I20260809 03:39:33.760741 2287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:33.760771 2300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:33.764518 2300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:33.771575 2300 catalog_manager.cc:1347] Generated new cluster ID: 8c20d33a2948475e88a16250646252e1
I20260809 03:39:33.771673 2300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:33.786535 2300 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:33.786926 2300 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:33.794840 2300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 298f874addaa45c1a615420b090720d4: Generated new TSK 0
I20260809 03:39:33.795672 2300 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:33.811111 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 2139
2026-08-09T03:39:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1671 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:33 dist-test-slave-jq4t krb5kdc[2319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:33 dist-test-slave-jq4t krb5kdc[2319](info): set up 2 sockets
Aug 09 03:39:33 dist-test-slave-jq4t krb5kdc[2319](info): commencing operation
krb5kdc: starting...
W20260809 03:39:35.093282 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:35 dist-test-slave-jq4t krb5kdc[2319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:39:35.232319 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:35.354244 2346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:35.354626 2346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:35.354707 2346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:35.358834 2346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:35.358984 2346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:35.359025 2346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:35.359067 2346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:35.359099 2346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:35.364629 2346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34267
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:35.365924 2346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:35.367082 2346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:35.376245 2358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:35.376268 2355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:35.376256 2356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:35.376494 2346 server_base.cc:1034] running on GCE node
I20260809 03:39:35.377367 2346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:35.378720 2346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:35.380060 2346 hybrid_clock.cc:648] HybridClock initialized: now 1786246775380009 us; error 57 us; skew 500 ppm
Aug 09 03:39:35 dist-test-slave-jq4t krb5kdc[2319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:35.384833 2346 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:39:35.386153 2346 webserver.cc:460] Webserver started at http://127.0.0.1:36419/ using document root <none> and password file <none>
I20260809 03:39:35.386993 2346 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:35.387122 2346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:35.387422 2346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:35.390503 2346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "6f0061a9a65d4a4cb26218be111f16ce"
format_stamp: "Formatted at 2026-08-09 03:39:35 on dist-test-slave-jq4t"
server_key: "9a81b80beab8d17a8175ba0db4393fe9"
server_key_iv: "c9561ad5078b0fde7df784b661777fa0"
server_key_version: "encryptionkey@0"
I20260809 03:39:35.391112 2346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "6f0061a9a65d4a4cb26218be111f16ce"
format_stamp: "Formatted at 2026-08-09 03:39:35 on dist-test-slave-jq4t"
server_key: "9a81b80beab8d17a8175ba0db4393fe9"
server_key_iv: "c9561ad5078b0fde7df784b661777fa0"
server_key_version: "encryptionkey@0"
I20260809 03:39:35.395284 2346 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:39:35.397929 2371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:35.399246 2346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:39:35.399396 2346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "6f0061a9a65d4a4cb26218be111f16ce"
format_stamp: "Formatted at 2026-08-09 03:39:35 on dist-test-slave-jq4t"
server_key: "9a81b80beab8d17a8175ba0db4393fe9"
server_key_iv: "c9561ad5078b0fde7df784b661777fa0"
server_key_version: "encryptionkey@0"
I20260809 03:39:35.399521 2346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:35.421132 2346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:35.424957 2346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:35.425185 2346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:35.432796 2346 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34267
I20260809 03:39:35.432814 2459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34267 every 8 connection(s)
I20260809 03:39:35.433846 2346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:35.436832 2460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:35.440783 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 2346
I20260809 03:39:35.440960 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:35.441295 1921 external_mini_cluster.cc:1428] Setting key b0ab9221c092fb50ab5f90279e1315c3
I20260809 03:39:35.443229 2460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce: Bootstrap starting.
I20260809 03:39:35.445616 2460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:35.446425 2460 log.cc:824] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:35.448411 2460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce: No bootstrap required, opened a new log
Aug 09 03:39:35 dist-test-slave-jq4t krb5kdc[2319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 03:39:35.452409 2460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f0061a9a65d4a4cb26218be111f16ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } }
I20260809 03:39:35.452698 2460 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:35.452773 2460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f0061a9a65d4a4cb26218be111f16ce, State: Initialized, Role: FOLLOWER
I20260809 03:39:35.453279 2460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f0061a9a65d4a4cb26218be111f16ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } }
I20260809 03:39:35.453408 2460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:35.453490 2460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:35.453606 2460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:35.454538 2460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f0061a9a65d4a4cb26218be111f16ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } }
I20260809 03:39:35.454891 2460 leader_election.cc:302] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f0061a9a65d4a4cb26218be111f16ce; no voters:
I20260809 03:39:35.455242 2460 leader_election.cc:288] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:35.455327 2471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:35.455560 2471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [term 1 LEADER]: Becoming Leader. State: Replica: 6f0061a9a65d4a4cb26218be111f16ce, State: Running, Role: LEADER
I20260809 03:39:35.455932 2471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f0061a9a65d4a4cb26218be111f16ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } }
I20260809 03:39:35.456682 2460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:35.456888 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:35.442889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:54891 (local address 127.0.0.1:34267)
0809 03:39:35.443416 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:35.443434 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:35.443476 (+ 42us) server_negotiation.cc:407] Connection header received
0809 03:39:35.444158 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:35.444177 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:35.444610 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:35.444953 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:35.445806 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:35.446920 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:35.447808 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:35.448302 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:35.448533 (+ 231us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:35.451447 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:35.451453 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:35.451469 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:35.451497 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:35.453489 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:35.454058 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:35.454065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:35.454071 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:35.454193 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:35.454550 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:35.454553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:35.454555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:35.455072 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:35.455334 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:35.455706 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 03:39:35.455976 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":149,"threads_started":1}
I20260809 03:39:35.457754 2473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f0061a9a65d4a4cb26218be111f16ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f0061a9a65d4a4cb26218be111f16ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } } }
I20260809 03:39:35.458159 2473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:35.458055 2472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f0061a9a65d4a4cb26218be111f16ce. Latest consensus state: current_term: 1 leader_uuid: "6f0061a9a65d4a4cb26218be111f16ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f0061a9a65d4a4cb26218be111f16ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } } }
I20260809 03:39:35.458551 2472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:35.459935 2489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:35.460765 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:35.463892 2489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:35.471470 2489 catalog_manager.cc:1347] Generated new cluster ID: a01d565325974e48ab801b3376aa6bf8
I20260809 03:39:35.471626 2489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:35.472842 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:35.463081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:33933 (local address 127.0.0.1:34267)
0809 03:39:35.463195 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:35.463201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:35.463403 (+ 202us) server_negotiation.cc:407] Connection header received
0809 03:39:35.463606 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:35.463609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:35.463661 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:35.463715 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:35.464802 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:35.465604 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:35.466604 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:35.466878 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:35.466997 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:35.468561 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:35.468567 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:35.468569 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:35.468598 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:35.470578 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:35.471223 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:35.471227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:35.471230 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:35.471322 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:35.471787 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:35.471791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:35.471793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:35.472022 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:35.472173 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:35.472563 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 03:39:35.472673 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":30}
I20260809 03:39:35.481611 2489 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:35.482554 2489 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:35.488778 2489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f0061a9a65d4a4cb26218be111f16ce: Generated new TSK 0
I20260809 03:39:35.489689 2489 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:35.499569 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 2346
W20260809 03:39:35.514961 2317 connection.cc:535] client connection to 127.0.0.1:34267 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34267): Connection reset by peer (error 104)
2026-08-09T03:39:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1674 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:35 dist-test-slave-jq4t krb5kdc[2508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:35 dist-test-slave-jq4t krb5kdc[2508](info): set up 2 sockets
Aug 09 03:39:35 dist-test-slave-jq4t krb5kdc[2508](info): commencing operation
krb5kdc: starting...
W20260809 03:39:36.764822 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:36 dist-test-slave-jq4t krb5kdc[2508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:39:36.899842 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35337
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:37.019469 2531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:37.019834 2531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:37.019915 2531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:37.025130 2531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:37.025251 2531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:37.025292 2531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:37.025327 2531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:37.025367 2531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:37.031711 2531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35337
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:37.033063 2531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:37.034191 2531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:37.043921 2538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:37.044128 2531 server_base.cc:1034] running on GCE node
W20260809 03:39:37.043952 2537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:37.043943 2541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:37.045009 2531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:37.046273 2531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:37.047458 2531 hybrid_clock.cc:648] HybridClock initialized: now 1786246777047443 us; error 33 us; skew 500 ppm
Aug 09 03:39:37 dist-test-slave-jq4t krb5kdc[2508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246777, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:37.051003 2531 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:39:37.052179 2531 webserver.cc:460] Webserver started at http://127.0.0.1:45631/ using document root <none> and password file <none>
I20260809 03:39:37.052896 2531 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:37.052990 2531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:37.053257 2531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:37.055529 2531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "d2026082f76546d7bb7bb9ebe6bf05b3"
format_stamp: "Formatted at 2026-08-09 03:39:37 on dist-test-slave-jq4t"
server_key: "ac147ca58311fbce16bcea1646c7c59a"
server_key_iv: "10c32a1cac7e4506a4d1bfac50ceecc6"
server_key_version: "encryptionkey@0"
I20260809 03:39:37.056152 2531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "d2026082f76546d7bb7bb9ebe6bf05b3"
format_stamp: "Formatted at 2026-08-09 03:39:37 on dist-test-slave-jq4t"
server_key: "ac147ca58311fbce16bcea1646c7c59a"
server_key_iv: "10c32a1cac7e4506a4d1bfac50ceecc6"
server_key_version: "encryptionkey@0"
I20260809 03:39:37.060218 2531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260809 03:39:37.062877 2552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:37.064123 2531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:39:37.064287 2531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "d2026082f76546d7bb7bb9ebe6bf05b3"
format_stamp: "Formatted at 2026-08-09 03:39:37 on dist-test-slave-jq4t"
server_key: "ac147ca58311fbce16bcea1646c7c59a"
server_key_iv: "10c32a1cac7e4506a4d1bfac50ceecc6"
server_key_version: "encryptionkey@0"
I20260809 03:39:37.064399 2531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:37.085881 2531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:37.089390 2531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:37.089625 2531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:37.097885 2531 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35337
I20260809 03:39:37.097908 2640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35337 every 8 connection(s)
I20260809 03:39:37.099071 2531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:37.102279 2642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:37.108525 2642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3: Bootstrap starting.
I20260809 03:39:37.108561 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 2531
I20260809 03:39:37.108705 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:37.109025 1921 external_mini_cluster.cc:1428] Setting key 863e568fa93bd1e43c96c03c6cedefb0
I20260809 03:39:37.111517 2642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:37.112334 2642 log.cc:824] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:37.114445 2642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3: No bootstrap required, opened a new log
I20260809 03:39:37.118598 2642 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2026082f76546d7bb7bb9ebe6bf05b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35337 } }
I20260809 03:39:37.118865 2642 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:37.118961 2642 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2026082f76546d7bb7bb9ebe6bf05b3, State: Initialized, Role: FOLLOWER
I20260809 03:39:37.119029 2648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:37.110626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:51061 (local address 127.0.0.1:35337)
0809 03:39:37.111172 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:37.111188 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:37.111226 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:39:37.112018 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:37.112041 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:37.112417 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:37.112860 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:37.113314 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:37.114344 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:37.115327 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:37.115823 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:37.116019 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:37.117073 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:37.117079 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:37.117094 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:37.117594 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:37.117655 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:37.117678 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:37.117829 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:37.118030 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 03:39:37.118186 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":187,"threads_started":1}
I20260809 03:39:37.119514 2642 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2026082f76546d7bb7bb9ebe6bf05b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35337 } }
I20260809 03:39:37.119686 2642 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:37.119755 2642 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:37.119853 2642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:37.120963 2642 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2026082f76546d7bb7bb9ebe6bf05b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35337 } }
I20260809 03:39:37.121335 2642 leader_election.cc:302] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2026082f76546d7bb7bb9ebe6bf05b3; no voters:
I20260809 03:39:37.121755 2642 leader_election.cc:288] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:37.121902 2653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:37.122133 2653 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [term 1 LEADER]: Becoming Leader. State: Replica: d2026082f76546d7bb7bb9ebe6bf05b3, State: Running, Role: LEADER
I20260809 03:39:37.122361 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:37.122536 2653 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2026082f76546d7bb7bb9ebe6bf05b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35337 } }
I20260809 03:39:37.123196 2642 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:37.124123 2655 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2026082f76546d7bb7bb9ebe6bf05b3. Latest consensus state: current_term: 1 leader_uuid: "d2026082f76546d7bb7bb9ebe6bf05b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2026082f76546d7bb7bb9ebe6bf05b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35337 } } }
I20260809 03:39:37.124246 2655 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:37.124625 2654 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2026082f76546d7bb7bb9ebe6bf05b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2026082f76546d7bb7bb9ebe6bf05b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35337 } } }
I20260809 03:39:37.124720 2654 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:37.125075 2672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:37.128669 2672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:37.131909 2648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:37.125707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:34331 (local address 127.0.0.1:35337)
0809 03:39:37.125841 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:37.125846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:37.126031 (+ 185us) server_negotiation.cc:407] Connection header received
0809 03:39:37.126176 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:37.126180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:37.126233 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:37.126304 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:37.126654 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:37.127436 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:37.129810 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:37.130177 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:37.130308 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:37.131345 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:37.131350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:37.131353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:37.131369 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:37.131401 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:37.131414 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:37.131514 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:37.131708 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 03:39:37.131750 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:39:37.135773 2672 catalog_manager.cc:1347] Generated new cluster ID: 6f780f200a5a40128903dfbbc250b377
I20260809 03:39:37.135869 2672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:37.148957 2672 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:37.149982 2672 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:37.155159 2672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2026082f76546d7bb7bb9ebe6bf05b3: Generated new TSK 0
I20260809 03:39:37.155936 2672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:37.161485 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 2531
W20260809 03:39:37.171306 2506 connection.cc:535] client connection to 127.0.0.1:35337 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35337): Connection reset by peer (error 104)
2026-08-09T03:39:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1656 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:37 dist-test-slave-jq4t krb5kdc[2692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:37 dist-test-slave-jq4t krb5kdc[2692](info): set up 2 sockets
Aug 09 03:39:37 dist-test-slave-jq4t krb5kdc[2692](info): commencing operation
krb5kdc: starting...
W20260809 03:39:38.414510 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:38 dist-test-slave-jq4t krb5kdc[2692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:39:38.551918 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:38.669370 2720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:38.669652 2720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:38.669695 2720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:38.673683 2720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:38.673751 2720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:38.673770 2720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:38.673787 2720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:38.673803 2720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:38.679167 2720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33715
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:38.680342 2720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:38.681550 2720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:38.690855 2726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:38.690874 2729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:38.690861 2727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:38.690922 2720 server_base.cc:1034] running on GCE node
I20260809 03:39:38.691727 2720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:38.693029 2720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:38.694254 2720 hybrid_clock.cc:648] HybridClock initialized: now 1786246778694223 us; error 41 us; skew 500 ppm
Aug 09 03:39:38 dist-test-slave-jq4t krb5kdc[2692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246778, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:38.697314 2720 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:39:38.698377 2720 webserver.cc:460] Webserver started at http://127.0.0.1:43123/ using document root <none> and password file <none>
I20260809 03:39:38.699033 2720 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:38.699136 2720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:38.699394 2720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:38.701813 2720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "24ffd66f43b7420ebf55325022752b5f"
format_stamp: "Formatted at 2026-08-09 03:39:38 on dist-test-slave-jq4t"
server_key: "6ce1dbb22408376f348022615434d0c5"
server_key_iv: "d378c65e4e07ef4c95bc2dbe1d70fc20"
server_key_version: "encryptionkey@0"
I20260809 03:39:38.702456 2720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "24ffd66f43b7420ebf55325022752b5f"
format_stamp: "Formatted at 2026-08-09 03:39:38 on dist-test-slave-jq4t"
server_key: "6ce1dbb22408376f348022615434d0c5"
server_key_iv: "d378c65e4e07ef4c95bc2dbe1d70fc20"
server_key_version: "encryptionkey@0"
I20260809 03:39:38.706524 2720 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:39:38.709110 2736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:38.710367 2720 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:39:38.710475 2720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "24ffd66f43b7420ebf55325022752b5f"
format_stamp: "Formatted at 2026-08-09 03:39:38 on dist-test-slave-jq4t"
server_key: "6ce1dbb22408376f348022615434d0c5"
server_key_iv: "d378c65e4e07ef4c95bc2dbe1d70fc20"
server_key_version: "encryptionkey@0"
I20260809 03:39:38.710565 2720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:38.715307 2720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:38.718391 2720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:38.718552 2720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:38.726627 2720 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33715
I20260809 03:39:38.726656 2825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33715 every 8 connection(s)
I20260809 03:39:38.727756 2720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:38.729606 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 2720
I20260809 03:39:38.729787 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:38.730163 1921 external_mini_cluster.cc:1428] Setting key 46cbf1980e221d451eaa084b7e1efaef
I20260809 03:39:38.731516 2826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:38.735900 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:38.731827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:58231 (local address 127.0.0.1:33715)
0809 03:39:38.732329 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:38.732346 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:38.732393 (+ 47us) server_negotiation.cc:407] Connection header received
0809 03:39:38.733122 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:38.733139 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:38.733427 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:38.733771 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:38.734663 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:38.734668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:38.734681 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:38.734692 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:38.734746 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:38.734766 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:38.734921 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:38.734996 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 03:39:38.735165 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":157,"threads_started":1}
I20260809 03:39:38.738426 2826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f: Bootstrap starting.
I20260809 03:39:38.739167 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:38.740963 2826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:38.741936 2826 log.cc:824] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:38.743237 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:38.741420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.48:37605 (local address 127.0.0.1:33715)
0809 03:39:38.741554 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:38.741558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:38.741719 (+ 161us) server_negotiation.cc:407] Connection header received
0809 03:39:38.741803 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:38.741808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:38.741846 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:38.741942 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:38.742772 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:38.742775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:38.742777 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:38.742788 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:38.742814 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:38.742824 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:38.742968 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:38.743075 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 03:39:38.743116 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:39:38.744470 2826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f: No bootstrap required, opened a new log
I20260809 03:39:38.748291 2826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ffd66f43b7420ebf55325022752b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33715 } }
I20260809 03:39:38.748600 2826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:38.748648 2826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24ffd66f43b7420ebf55325022752b5f, State: Initialized, Role: FOLLOWER
I20260809 03:39:38.749078 2826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ffd66f43b7420ebf55325022752b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33715 } }
I20260809 03:39:38.749205 2826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:38.749248 2826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:38.749322 2826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:38.750291 2826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ffd66f43b7420ebf55325022752b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33715 } }
I20260809 03:39:38.750612 2826 leader_election.cc:302] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24ffd66f43b7420ebf55325022752b5f; no voters:
I20260809 03:39:38.750967 2826 leader_election.cc:288] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:38.751147 2843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:38.751410 2843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [term 1 LEADER]: Becoming Leader. State: Replica: 24ffd66f43b7420ebf55325022752b5f, State: Running, Role: LEADER
I20260809 03:39:38.751770 2843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ffd66f43b7420ebf55325022752b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33715 } }
I20260809 03:39:38.752344 2826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:38.753654 2847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24ffd66f43b7420ebf55325022752b5f. Latest consensus state: current_term: 1 leader_uuid: "24ffd66f43b7420ebf55325022752b5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ffd66f43b7420ebf55325022752b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33715 } } }
I20260809 03:39:38.753711 2845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24ffd66f43b7420ebf55325022752b5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ffd66f43b7420ebf55325022752b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33715 } } }
I20260809 03:39:38.753803 2847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:38.753826 2845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:38.754233 2865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:38.757280 2865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:38.763501 2865 catalog_manager.cc:1347] Generated new cluster ID: d203d268098e4c1cb021f064b930755e
I20260809 03:39:38.763597 2865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:38.778218 2865 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:38.778656 2865 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:38.787951 2865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24ffd66f43b7420ebf55325022752b5f: Generated new TSK 0
I20260809 03:39:38.788872 2865 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:38.800056 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 2720
2026-08-09T03:39:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1637 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:38 dist-test-slave-jq4t krb5kdc[2884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:38 dist-test-slave-jq4t krb5kdc[2884](info): set up 2 sockets
Aug 09 03:39:38 dist-test-slave-jq4t krb5kdc[2884](info): commencing operation
krb5kdc: starting...
W20260809 03:39:40.057456 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:40 dist-test-slave-jq4t krb5kdc[2884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:39:40.192003 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:44081
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:35221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:44081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:40.310999 2904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:40.311290 2904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:40.311365 2904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:40.315500 2904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:40.315582 2904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:40.315601 2904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:40.315618 2904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:40.315634 2904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:40.321496 2904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:35221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:44081
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:44081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:40.322770 2904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:40.323927 2904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:40.333264 2917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:40.333297 2922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:40.333264 2916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:40.333726 2904 server_base.cc:1034] running on GCE node
I20260809 03:39:40.334327 2904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:40.335449 2904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:40.336632 2904 hybrid_clock.cc:648] HybridClock initialized: now 1786246780336611 us; error 31 us; skew 500 ppm
Aug 09 03:39:40 dist-test-slave-jq4t krb5kdc[2884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246780, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:40.339514 2904 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:39:40.340469 2904 webserver.cc:460] Webserver started at http://127.1.224.126:43187/ using document root <none> and password file <none>
I20260809 03:39:40.341154 2904 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:40.341252 2904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:40.341516 2904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:40.343736 2904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "0632cf120333465792bf056717e236a1"
format_stamp: "Formatted at 2026-08-09 03:39:40 on dist-test-slave-jq4t"
server_key: "f58b463d799c8346df60b29c36bb8a4a"
server_key_iv: "98721521609acef7a04535692b2ad8d0"
server_key_version: "encryptionkey@0"
I20260809 03:39:40.344341 2904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "0632cf120333465792bf056717e236a1"
format_stamp: "Formatted at 2026-08-09 03:39:40 on dist-test-slave-jq4t"
server_key: "f58b463d799c8346df60b29c36bb8a4a"
server_key_iv: "98721521609acef7a04535692b2ad8d0"
server_key_version: "encryptionkey@0"
I20260809 03:39:40.348296 2904 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260809 03:39:40.351004 2930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:40.352380 2904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:39:40.352588 2904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "0632cf120333465792bf056717e236a1"
format_stamp: "Formatted at 2026-08-09 03:39:40 on dist-test-slave-jq4t"
server_key: "f58b463d799c8346df60b29c36bb8a4a"
server_key_iv: "98721521609acef7a04535692b2ad8d0"
server_key_version: "encryptionkey@0"
I20260809 03:39:40.352725 2904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:40.369650 2904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:40.372951 2904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:40.373198 2904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:40.381136 2904 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:44081
I20260809 03:39:40.381155 3015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:44081 every 8 connection(s)
I20260809 03:39:40.382367 2904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:40.385715 3016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:40.390343 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 2904
I20260809 03:39:40.390489 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:40.390784 1921 external_mini_cluster.cc:1428] Setting key dfa16c1753b6a96cf54a98b61c91a060
I20260809 03:39:40.392354 3016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1: Bootstrap starting.
I20260809 03:39:40.394833 3016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:40.395663 3016 log.cc:824] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:40.397825 3016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1: No bootstrap required, opened a new log
Aug 09 03:39:40 dist-test-slave-jq4t krb5kdc[2884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:40.402359 3016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0632cf120333465792bf056717e236a1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44081 } }
I20260809 03:39:40.402637 3016 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:40.402705 3016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0632cf120333465792bf056717e236a1, State: Initialized, Role: FOLLOWER
I20260809 03:39:40.403198 3016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0632cf120333465792bf056717e236a1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44081 } }
I20260809 03:39:40.403369 3016 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:40.403453 3016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:40.403584 3016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:40.404808 3016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0632cf120333465792bf056717e236a1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44081 } }
I20260809 03:39:40.405179 3016 leader_election.cc:302] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0632cf120333465792bf056717e236a1; no voters:
I20260809 03:39:40.405551 3016 leader_election.cc:288] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:40.405954 3034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:40.406229 3034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [term 1 LEADER]: Becoming Leader. State: Replica: 0632cf120333465792bf056717e236a1, State: Running, Role: LEADER
I20260809 03:39:40.406571 3034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0632cf120333465792bf056717e236a1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44081 } }
I20260809 03:39:40.406786 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:40.392418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40721 (local address 127.1.224.126:44081)
0809 03:39:40.392958 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:40.392974 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:40.393014 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:39:40.393700 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:40.393719 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:40.394074 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:40.394432 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:40.395354 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:40.396525 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:40.397445 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:40.397976 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:40.398186 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:40.401106 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:40.401112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:40.401128 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:40.401159 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:40.403170 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:40.403817 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:40.403822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:40.403827 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:40.403942 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:40.404342 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:40.404344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:40.404345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:40.404846 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:40.405098 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:40.405432 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 03:39:40.405690 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":164,"threads_started":1}
I20260809 03:39:40.406914 3016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:40.407856 3034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0632cf120333465792bf056717e236a1. Latest consensus state: current_term: 1 leader_uuid: "0632cf120333465792bf056717e236a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0632cf120333465792bf056717e236a1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44081 } } }
I20260809 03:39:40.407987 3034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:40.408444 3044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:40.408457 3032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0632cf120333465792bf056717e236a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0632cf120333465792bf056717e236a1" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 44081 } } }
I20260809 03:39:40.408735 3032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:40.410509 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:40.411755 3044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:40.418089 3044 catalog_manager.cc:1347] Generated new cluster ID: ce90c264deeb42f49b733b40f968ff9b
I20260809 03:39:40.418164 3044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:40.427273 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:40.414399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45137 (local address 127.1.224.126:44081)
0809 03:39:40.414515 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:40.414519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:40.416023 (+ 1504us) server_negotiation.cc:407] Connection header received
0809 03:39:40.416192 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:40.416196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:40.416255 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:40.416345 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:40.417109 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:40.417662 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:40.418803 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:40.419186 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:40.419303 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:40.423486 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:40.423491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:40.423494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:40.423524 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:40.425330 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:40.425903 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:40.425907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:40.425909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:40.426011 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:40.426339 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:40.426342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:40.426344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:40.426568 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:40.426708 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:40.426987 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 03:39:40.427092 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:39:40.446345 3044 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:40.447407 3044 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:40.456446 3044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0632cf120333465792bf056717e236a1: Generated new TSK 0
I20260809 03:39:40.457289 3044 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:40.465835 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 2904
W20260809 03:39:40.479220 2882 connection.cc:535] client connection to 127.1.224.126:44081 recv error: Network error: failed to read from TLS socket (remote: 127.1.224.126:44081): Connection reset by peer (error 104)
2026-08-09T03:39:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1670 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:40 dist-test-slave-jq4t krb5kdc[3073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:40 dist-test-slave-jq4t krb5kdc[3073](info): set up 2 sockets
Aug 09 03:39:40 dist-test-slave-jq4t krb5kdc[3073](info): commencing operation
krb5kdc: starting...
W20260809 03:39:41.725095 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:41 dist-test-slave-jq4t krb5kdc[3073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:39:41.859336 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:43415
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:43415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:41.979832 3095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:41.980206 3095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:41.980309 3095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:41.984498 3095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:41.984613 3095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:41.984653 3095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:41.984690 3095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:41.984725 3095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:41.990337 3095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:43415
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:43415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:41.991608 3095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:41.992786 3095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:42.002084 3105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:42.002151 3108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:42.002069 3102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:42.002228 3095 server_base.cc:1034] running on GCE node
I20260809 03:39:42.002938 3095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:42.004103 3095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:42.005302 3095 hybrid_clock.cc:648] HybridClock initialized: now 1786246782005279 us; error 35 us; skew 500 ppm
Aug 09 03:39:42 dist-test-slave-jq4t krb5kdc[3073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246782, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:42.008352 3095 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:39:42.009305 3095 webserver.cc:460] Webserver started at http://127.1.224.126:33477/ using document root <none> and password file <none>
I20260809 03:39:42.009909 3095 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:42.010004 3095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:42.010254 3095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:42.012372 3095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "2a9f65ffac324afb87d044c89d0004cf"
format_stamp: "Formatted at 2026-08-09 03:39:42 on dist-test-slave-jq4t"
server_key: "c610d9a72b99db6adc8db00b4f021290"
server_key_iv: "a18d9b218ea225d8ca4a6410bd908ffd"
server_key_version: "encryptionkey@0"
I20260809 03:39:42.013032 3095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "2a9f65ffac324afb87d044c89d0004cf"
format_stamp: "Formatted at 2026-08-09 03:39:42 on dist-test-slave-jq4t"
server_key: "c610d9a72b99db6adc8db00b4f021290"
server_key_iv: "a18d9b218ea225d8ca4a6410bd908ffd"
server_key_version: "encryptionkey@0"
I20260809 03:39:42.016964 3095 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:39:42.019433 3124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:42.020798 3095 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:39:42.020954 3095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "2a9f65ffac324afb87d044c89d0004cf"
format_stamp: "Formatted at 2026-08-09 03:39:42 on dist-test-slave-jq4t"
server_key: "c610d9a72b99db6adc8db00b4f021290"
server_key_iv: "a18d9b218ea225d8ca4a6410bd908ffd"
server_key_version: "encryptionkey@0"
I20260809 03:39:42.021073 3095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:42.036991 3095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:42.040009 3095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:42.040225 3095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:42.047734 3095 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:43415
I20260809 03:39:42.047746 3216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:43415 every 8 connection(s)
I20260809 03:39:42.048843 3095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:42.051774 3219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:42.057575 3219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf: Bootstrap starting.
I20260809 03:39:42.057749 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 3095
I20260809 03:39:42.057895 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:42.058190 1921 external_mini_cluster.cc:1428] Setting key ec3af38d01b3f140f6a79a21652838ba
I20260809 03:39:42.060582 3219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:42.061525 3219 log.cc:824] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:42.064018 3219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf: No bootstrap required, opened a new log
I20260809 03:39:42.068763 3219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a9f65ffac324afb87d044c89d0004cf" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43415 } }
I20260809 03:39:42.069046 3219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:42.069108 3219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9f65ffac324afb87d044c89d0004cf, State: Initialized, Role: FOLLOWER
I20260809 03:39:42.069332 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:42.059664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43273 (local address 127.1.224.126:43415)
0809 03:39:42.060133 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:42.060152 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:42.060196 (+ 44us) server_negotiation.cc:407] Connection header received
0809 03:39:42.061151 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:42.061176 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:42.061629 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:42.062020 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:42.062488 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:42.064013 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:42.065137 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:42.065829 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:42.066009 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:42.066925 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:42.066932 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:42.066954 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:42.067551 (+ 597us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:42.067623 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:42.067652 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:42.067791 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:42.068153 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 03:39:42.068302 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":113,"threads_started":1}
I20260809 03:39:42.069624 3219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a9f65ffac324afb87d044c89d0004cf" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43415 } }
I20260809 03:39:42.069777 3219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:42.069868 3219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:42.069977 3219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:42.070966 3219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a9f65ffac324afb87d044c89d0004cf" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43415 } }
I20260809 03:39:42.071346 3219 leader_election.cc:302] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a9f65ffac324afb87d044c89d0004cf; no voters:
I20260809 03:39:42.071683 3219 leader_election.cc:288] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:42.071805 3226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:42.072091 3226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [term 1 LEADER]: Becoming Leader. State: Replica: 2a9f65ffac324afb87d044c89d0004cf, State: Running, Role: LEADER
I20260809 03:39:42.072412 3226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a9f65ffac324afb87d044c89d0004cf" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43415 } }
I20260809 03:39:42.072664 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:42.073091 3219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:42.074072 3227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a9f65ffac324afb87d044c89d0004cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a9f65ffac324afb87d044c89d0004cf" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43415 } } }
I20260809 03:39:42.074187 3227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:42.074425 3228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a9f65ffac324afb87d044c89d0004cf. Latest consensus state: current_term: 1 leader_uuid: "2a9f65ffac324afb87d044c89d0004cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a9f65ffac324afb87d044c89d0004cf" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 43415 } } }
I20260809 03:39:42.074512 3228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:42.074770 3250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:42.078948 3250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:42.080755 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:42.075737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46397 (local address 127.1.224.126:43415)
0809 03:39:42.075888 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:42.075893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:42.076200 (+ 307us) server_negotiation.cc:407] Connection header received
0809 03:39:42.076317 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:42.076320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:42.076376 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:42.076460 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:42.076809 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:42.077576 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:42.078765 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:42.079113 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:42.079227 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:42.080160 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:42.080164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:42.080167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:42.080182 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:42.080215 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:42.080229 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:42.080328 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:42.080553 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 03:39:42.080593 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:39:42.085429 3250 catalog_manager.cc:1347] Generated new cluster ID: e7d63af30feb452aa67f52f24a306476
I20260809 03:39:42.085515 3250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:42.094696 3250 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:42.096235 3250 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:42.102021 3250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a9f65ffac324afb87d044c89d0004cf: Generated new TSK 0
I20260809 03:39:42.102900 3250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:42.112303 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 3095
W20260809 03:39:42.123662 3071 connection.cc:535] client connection to 127.1.224.126:43415 recv error: Network error: failed to read from TLS socket (remote: 127.1.224.126:43415): Connection reset by peer (error 104)
2026-08-09T03:39:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1644 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:42 dist-test-slave-jq4t krb5kdc[3272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:42 dist-test-slave-jq4t krb5kdc[3272](info): set up 2 sockets
Aug 09 03:39:42 dist-test-slave-jq4t krb5kdc[3272](info): commencing operation
krb5kdc: starting...
W20260809 03:39:43.366405 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:43 dist-test-slave-jq4t krb5kdc[3272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:39:43.501662 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:39795
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:36621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:39795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:43.618479 3293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:43.618821 3293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:43.618924 3293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:43.623061 3293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:43.623155 3293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:43.623194 3293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:43.623252 3293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:43.623291 3293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:43.628762 3293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:36621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:39795
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:39795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:43.629956 3293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:43.631094 3293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:43.640301 3300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:43.640288 3304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:43.640513 3293 server_base.cc:1034] running on GCE node
W20260809 03:39:43.640288 3299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:43.641309 3293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:43.642524 3293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:43.643711 3293 hybrid_clock.cc:648] HybridClock initialized: now 1786246783643702 us; error 32 us; skew 500 ppm
Aug 09 03:39:43 dist-test-slave-jq4t krb5kdc[3272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246783, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:43.646692 3293 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:39:43.648073 3293 webserver.cc:460] Webserver started at http://127.1.224.126:42635/ using document root <none> and password file <none>
I20260809 03:39:43.648741 3293 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:43.648836 3293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:43.649066 3293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:43.651228 3293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "513f5e47cba64782933bd87d09c12397"
format_stamp: "Formatted at 2026-08-09 03:39:43 on dist-test-slave-jq4t"
server_key: "90070e5f229e93bfd19ec74d45bf44c5"
server_key_iv: "e20e100e67729c9d5aeee089b262a45f"
server_key_version: "encryptionkey@0"
I20260809 03:39:43.651813 3293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "513f5e47cba64782933bd87d09c12397"
format_stamp: "Formatted at 2026-08-09 03:39:43 on dist-test-slave-jq4t"
server_key: "90070e5f229e93bfd19ec74d45bf44c5"
server_key_iv: "e20e100e67729c9d5aeee089b262a45f"
server_key_version: "encryptionkey@0"
I20260809 03:39:43.655707 3293 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:39:43.658156 3314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:43.659462 3293 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:39:43.659606 3293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "513f5e47cba64782933bd87d09c12397"
format_stamp: "Formatted at 2026-08-09 03:39:43 on dist-test-slave-jq4t"
server_key: "90070e5f229e93bfd19ec74d45bf44c5"
server_key_iv: "e20e100e67729c9d5aeee089b262a45f"
server_key_version: "encryptionkey@0"
I20260809 03:39:43.659718 3293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:43.664412 3293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:43.667708 3293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:43.667905 3293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:43.675480 3293 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:39795
I20260809 03:39:43.675518 3406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:39795 every 8 connection(s)
I20260809 03:39:43.676462 3293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:43.678972 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 3293
I20260809 03:39:43.679126 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:43.679467 1921 external_mini_cluster.cc:1428] Setting key ba2d247508b4b995fbb4ed676f956eef
I20260809 03:39:43.679694 3407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:43.685320 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:43.681002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53305 (local address 127.1.224.126:39795)
0809 03:39:43.681459 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:43.681476 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:43.681522 (+ 46us) server_negotiation.cc:407] Connection header received
0809 03:39:43.682488 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:43.682516 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:43.682804 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:43.683157 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:43.684020 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:43.684025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:43.684037 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:43.684048 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:43.684105 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:43.684124 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:43.684279 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:43.684351 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 03:39:43.684554 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":148,"threads_started":1}
I20260809 03:39:43.686832 3407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397: Bootstrap starting.
I20260809 03:39:43.688210 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:43.689457 3407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:43.690292 3407 log.cc:824] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:43.692337 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:43.690509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58321 (local address 127.1.224.126:39795)
0809 03:39:43.690650 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:43.690654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:43.690880 (+ 226us) server_negotiation.cc:407] Connection header received
0809 03:39:43.691046 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:43.691049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:43.691089 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:43.691179 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:43.691928 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:43.691932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:43.691935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:43.691946 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:43.691978 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:43.691994 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:43.692095 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:43.692168 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 03:39:43.692212 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:39:43.692761 3407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397: No bootstrap required, opened a new log
I20260809 03:39:43.696990 3407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513f5e47cba64782933bd87d09c12397" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39795 } }
I20260809 03:39:43.697273 3407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:43.697356 3407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513f5e47cba64782933bd87d09c12397, State: Initialized, Role: FOLLOWER
I20260809 03:39:43.697816 3407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513f5e47cba64782933bd87d09c12397" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39795 } }
I20260809 03:39:43.697988 3407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:43.698079 3407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:43.698207 3407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:43.699234 3407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513f5e47cba64782933bd87d09c12397" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39795 } }
I20260809 03:39:43.699608 3407 leader_election.cc:302] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 513f5e47cba64782933bd87d09c12397; no voters:
I20260809 03:39:43.699965 3407 leader_election.cc:288] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:43.700107 3427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:43.700363 3427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [term 1 LEADER]: Becoming Leader. State: Replica: 513f5e47cba64782933bd87d09c12397, State: Running, Role: LEADER
I20260809 03:39:43.700730 3427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513f5e47cba64782933bd87d09c12397" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39795 } }
I20260809 03:39:43.701431 3407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:43.702683 3430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 513f5e47cba64782933bd87d09c12397. Latest consensus state: current_term: 1 leader_uuid: "513f5e47cba64782933bd87d09c12397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513f5e47cba64782933bd87d09c12397" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39795 } } }
I20260809 03:39:43.702678 3429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "513f5e47cba64782933bd87d09c12397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513f5e47cba64782933bd87d09c12397" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 39795 } } }
I20260809 03:39:43.702831 3430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:43.702831 3429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:43.703316 3442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:43.706290 3442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:43.712164 3442 catalog_manager.cc:1347] Generated new cluster ID: ce4d0464fe6142b097712bcec6dc1996
I20260809 03:39:43.712241 3442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:43.722261 3442 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:43.722652 3442 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:43.726924 3442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 513f5e47cba64782933bd87d09c12397: Generated new TSK 0
I20260809 03:39:43.727695 3442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:43.737813 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 3293
2026-08-09T03:39:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1632 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:43 dist-test-slave-jq4t krb5kdc[3464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:43 dist-test-slave-jq4t krb5kdc[3464](info): set up 2 sockets
Aug 09 03:39:43 dist-test-slave-jq4t krb5kdc[3464](info): commencing operation
krb5kdc: starting...
W20260809 03:39:45.005316 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:45 dist-test-slave-jq4t krb5kdc[3464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:39:45.139987 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37623
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:43187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:37623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:45.258140 3484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:45.258430 3484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:45.258474 3484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:45.262398 3484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:45.262459 3484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:45.262478 3484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:45.262496 3484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:45.262511 3484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:45.268436 3484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:43187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:37623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:37623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:45.269690 3484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:45.270849 3484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:45.280248 3493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:45.280289 3495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:45.280606 3484 server_base.cc:1034] running on GCE node
W20260809 03:39:45.280289 3500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:45.281319 3484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:45.282483 3484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:45.283661 3484 hybrid_clock.cc:648] HybridClock initialized: now 1786246785283640 us; error 33 us; skew 500 ppm
Aug 09 03:39:45 dist-test-slave-jq4t krb5kdc[3464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246785, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:45.286931 3484 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:39:45.288053 3484 webserver.cc:460] Webserver started at http://127.1.224.126:41785/ using document root <none> and password file <none>
I20260809 03:39:45.288847 3484 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:45.288967 3484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:45.289291 3484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:45.291654 3484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "f2415dd0e5c7405a9bbf4f27e2c55390"
format_stamp: "Formatted at 2026-08-09 03:39:45 on dist-test-slave-jq4t"
server_key: "118ec7b22f8fd2350e3d6ab544080044"
server_key_iv: "f3dacec03ce6986fdd8ca36d9aba2221"
server_key_version: "encryptionkey@0"
I20260809 03:39:45.292305 3484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "f2415dd0e5c7405a9bbf4f27e2c55390"
format_stamp: "Formatted at 2026-08-09 03:39:45 on dist-test-slave-jq4t"
server_key: "118ec7b22f8fd2350e3d6ab544080044"
server_key_iv: "f3dacec03ce6986fdd8ca36d9aba2221"
server_key_version: "encryptionkey@0"
I20260809 03:39:45.296326 3484 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 03:39:45.299016 3513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:45.300273 3484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:39:45.300432 3484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "f2415dd0e5c7405a9bbf4f27e2c55390"
format_stamp: "Formatted at 2026-08-09 03:39:45 on dist-test-slave-jq4t"
server_key: "118ec7b22f8fd2350e3d6ab544080044"
server_key_iv: "f3dacec03ce6986fdd8ca36d9aba2221"
server_key_version: "encryptionkey@0"
I20260809 03:39:45.300614 3484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:45.323720 3484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:45.327368 3484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:45.327603 3484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:45.335150 3484 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:37623
I20260809 03:39:45.335237 3602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:37623 every 8 connection(s)
I20260809 03:39:45.336177 3484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:45.338366 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 3484
I20260809 03:39:45.338518 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:45.338821 1921 external_mini_cluster.cc:1428] Setting key 3ba4ed9805a5f81f2417409f6e222a6e
I20260809 03:39:45.339280 3604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:45.346964 3604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390: Bootstrap starting.
Aug 09 03:39:45 dist-test-slave-jq4t krb5kdc[3464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:45.349684 3604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:45.350471 3604 log.cc:824] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:45.352800 3604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390: No bootstrap required, opened a new log
I20260809 03:39:45.354606 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:45.340350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44249 (local address 127.1.224.126:37623)
0809 03:39:45.340867 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:45.340889 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:45.340932 (+ 43us) server_negotiation.cc:407] Connection header received
0809 03:39:45.341760 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:45.341784 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:45.342227 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:45.342621 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:45.343310 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:45.344877 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:45.345844 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:45.346561 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:45.346730 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:45.349678 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:45.349684 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:45.349699 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:45.349749 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:45.351615 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:45.352133 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:45.352138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:45.352142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:45.352255 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:45.352580 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:45.352582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:45.352583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:45.353046 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:45.353206 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:45.353519 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 03:39:45.353853 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":188,"threads_started":1}
I20260809 03:39:45.356705 3604 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2415dd0e5c7405a9bbf4f27e2c55390" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37623 } }
I20260809 03:39:45.357039 3604 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:45.357124 3604 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2415dd0e5c7405a9bbf4f27e2c55390, State: Initialized, Role: FOLLOWER
I20260809 03:39:45.357641 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:45.357599 3604 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2415dd0e5c7405a9bbf4f27e2c55390" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37623 } }
I20260809 03:39:45.357769 3604 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:45.357846 3604 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:45.357995 3604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:45.358968 3604 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2415dd0e5c7405a9bbf4f27e2c55390" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37623 } }
I20260809 03:39:45.359329 3604 leader_election.cc:302] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2415dd0e5c7405a9bbf4f27e2c55390; no voters:
I20260809 03:39:45.359671 3604 leader_election.cc:288] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:45.359761 3616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:45.359939 3616 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [term 1 LEADER]: Becoming Leader. State: Replica: f2415dd0e5c7405a9bbf4f27e2c55390, State: Running, Role: LEADER
I20260809 03:39:45.360225 3616 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2415dd0e5c7405a9bbf4f27e2c55390" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37623 } }
I20260809 03:39:45.361037 3604 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:45.362195 3618 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2415dd0e5c7405a9bbf4f27e2c55390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2415dd0e5c7405a9bbf4f27e2c55390" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37623 } } }
I20260809 03:39:45.362334 3618 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:45.362680 3619 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2415dd0e5c7405a9bbf4f27e2c55390. Latest consensus state: current_term: 1 leader_uuid: "f2415dd0e5c7405a9bbf4f27e2c55390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2415dd0e5c7405a9bbf4f27e2c55390" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 37623 } } }
I20260809 03:39:45.362735 3631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:45.362802 3619 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:45.366228 3631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:45.370260 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:45.359918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36941 (local address 127.1.224.126:37623)
0809 03:39:45.360060 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:45.360066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:45.360361 (+ 295us) server_negotiation.cc:407] Connection header received
0809 03:39:45.360600 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:45.360604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:45.360648 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:45.360700 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:45.361641 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:45.362579 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:45.363944 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:45.364295 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:45.364403 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:45.366423 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:45.366428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:45.366430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:45.366457 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:45.368241 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:45.368817 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:45.368822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:45.368825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:45.368917 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:45.369244 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:45.369249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:45.369251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:45.369489 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:45.369630 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:45.370036 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 03:39:45.370145 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:39:45.373893 3631 catalog_manager.cc:1347] Generated new cluster ID: 650f3c9654bb419cab31bc298f88e1ad
I20260809 03:39:45.373984 3631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:45.389751 3631 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:45.391217 3631 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:45.405902 3631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2415dd0e5c7405a9bbf4f27e2c55390: Generated new TSK 0
I20260809 03:39:45.406708 3631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:45.411625 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 3484
2026-08-09T03:39:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1674 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:45 dist-test-slave-jq4t krb5kdc[3648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:45 dist-test-slave-jq4t krb5kdc[3648](info): set up 2 sockets
Aug 09 03:39:45 dist-test-slave-jq4t krb5kdc[3648](info): commencing operation
krb5kdc: starting...
W20260809 03:39:46.733909 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:46 dist-test-slave-jq4t krb5kdc[3648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:39:46.867125 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:33299
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:33299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:46.987316 3671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:46.987704 3671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:46.987792 3671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:46.992019 3671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:46.992154 3671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:46.992192 3671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:46.992237 3671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:46.992274 3671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:46.997968 3671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:33299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:33299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:46.999266 3671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:47.000416 3671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:47.009621 3689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:47.009635 3691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:47.009707 3686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:47.009963 3671 server_base.cc:1034] running on GCE node
I20260809 03:39:47.010726 3671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:47.011925 3671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:47.013144 3671 hybrid_clock.cc:648] HybridClock initialized: now 1786246787013112 us; error 52 us; skew 500 ppm
Aug 09 03:39:47 dist-test-slave-jq4t krb5kdc[3648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246787, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:47.016150 3671 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:39:47.017243 3671 webserver.cc:460] Webserver started at http://127.1.224.126:40163/ using document root <none> and password file <none>
I20260809 03:39:47.017885 3671 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:47.017987 3671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:47.018250 3671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:47.020658 3671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "a856baddcb794e20b0b12e04534bff88"
format_stamp: "Formatted at 2026-08-09 03:39:47 on dist-test-slave-jq4t"
server_key: "cb46dd4fca8d35cde2b26380608cc607"
server_key_iv: "b1722a467791b5ebde4338813edde323"
server_key_version: "encryptionkey@0"
I20260809 03:39:47.021274 3671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "a856baddcb794e20b0b12e04534bff88"
format_stamp: "Formatted at 2026-08-09 03:39:47 on dist-test-slave-jq4t"
server_key: "cb46dd4fca8d35cde2b26380608cc607"
server_key_iv: "b1722a467791b5ebde4338813edde323"
server_key_version: "encryptionkey@0"
I20260809 03:39:47.025400 3671 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:39:47.028019 3705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:47.029359 3671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:39:47.029517 3671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "a856baddcb794e20b0b12e04534bff88"
format_stamp: "Formatted at 2026-08-09 03:39:47 on dist-test-slave-jq4t"
server_key: "cb46dd4fca8d35cde2b26380608cc607"
server_key_iv: "b1722a467791b5ebde4338813edde323"
server_key_version: "encryptionkey@0"
I20260809 03:39:47.029649 3671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:47.068312 3671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:47.071822 3671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:47.072083 3671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:47.080332 3671 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:33299
I20260809 03:39:47.080359 3795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:33299 every 8 connection(s)
I20260809 03:39:47.081444 3671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:47.084461 3797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:47.085572 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 3671
I20260809 03:39:47.085717 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:47.086022 1921 external_mini_cluster.cc:1428] Setting key e16cf765e0a71fe7c89849aa4aa6ec2d
I20260809 03:39:47.091794 3797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88: Bootstrap starting.
I20260809 03:39:47.094718 3797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:47.095703 3797 log.cc:824] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:47.096884 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:47.087593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57633 (local address 127.1.224.126:33299)
0809 03:39:47.088072 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:47.088094 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:47.088144 (+ 50us) server_negotiation.cc:407] Connection header received
0809 03:39:47.089009 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:47.089035 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:47.089479 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:47.089862 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:47.090309 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:47.091866 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:47.092902 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:47.093630 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:47.093806 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:47.094736 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:47.094745 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:47.094763 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:47.095302 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:47.095373 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:47.095402 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:47.095522 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:47.095636 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 03:39:47.095867 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":122,"threads_started":1}
I20260809 03:39:47.098167 3797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88: No bootstrap required, opened a new log
I20260809 03:39:47.099884 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:47.102285 3797 raft_consensus.cc:348] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a856baddcb794e20b0b12e04534bff88" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33299 } }
I20260809 03:39:47.102596 3797 raft_consensus.cc:374] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:47.102674 3797 raft_consensus.cc:729] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a856baddcb794e20b0b12e04534bff88, State: Initialized, Role: FOLLOWER
I20260809 03:39:47.103175 3797 consensus_queue.cc:260] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a856baddcb794e20b0b12e04534bff88" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33299 } }
I20260809 03:39:47.103313 3797 raft_consensus.cc:388] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:47.103403 3797 raft_consensus.cc:482] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:47.103530 3797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:47.104812 3797 raft_consensus.cc:504] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a856baddcb794e20b0b12e04534bff88" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33299 } }
I20260809 03:39:47.105207 3797 leader_election.cc:302] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a856baddcb794e20b0b12e04534bff88; no voters:
I20260809 03:39:47.105609 3797 leader_election.cc:288] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:47.105710 3811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:47.105997 3811 raft_consensus.cc:686] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [term 1 LEADER]: Becoming Leader. State: Replica: a856baddcb794e20b0b12e04534bff88, State: Running, Role: LEADER
I20260809 03:39:47.106393 3811 consensus_queue.cc:237] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a856baddcb794e20b0b12e04534bff88" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33299 } }
I20260809 03:39:47.106760 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:47.102082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51667 (local address 127.1.224.126:33299)
0809 03:39:47.102224 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:47.102229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:47.102440 (+ 211us) server_negotiation.cc:407] Connection header received
0809 03:39:47.102599 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:47.102603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:47.102675 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:47.102761 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:47.103036 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:47.103762 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:47.104901 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:47.105198 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:47.105282 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:47.106271 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:47.106275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:47.106278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:47.106292 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:47.106321 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:47.106342 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:47.106388 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:47.106574 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 03:39:47.106633 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:39:47.107203 3797 sys_catalog.cc:564] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:47.108266 3813 sys_catalog.cc:455] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a856baddcb794e20b0b12e04534bff88. Latest consensus state: current_term: 1 leader_uuid: "a856baddcb794e20b0b12e04534bff88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a856baddcb794e20b0b12e04534bff88" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33299 } } }
I20260809 03:39:47.108408 3813 sys_catalog.cc:458] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:47.108618 3812 sys_catalog.cc:455] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a856baddcb794e20b0b12e04534bff88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a856baddcb794e20b0b12e04534bff88" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33299 } } }
I20260809 03:39:47.108712 3812 sys_catalog.cc:458] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:47.108865 3828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:47.111927 3828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:47.117671 3828 catalog_manager.cc:1347] Generated new cluster ID: 9960effacf3246fa8e89529575c1da16
I20260809 03:39:47.117812 3828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:47.130265 3828 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:47.131330 3828 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:47.136071 3828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a856baddcb794e20b0b12e04534bff88: Generated new TSK 0
I20260809 03:39:47.136858 3828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:47.146271 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 3671
2026-08-09T03:39:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1732 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:47 dist-test-slave-jq4t krb5kdc[3844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:47 dist-test-slave-jq4t krb5kdc[3844](info): set up 2 sockets
Aug 09 03:39:47 dist-test-slave-jq4t krb5kdc[3844](info): commencing operation
krb5kdc: starting...
W20260809 03:39:48.408279 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:48 dist-test-slave-jq4t krb5kdc[3844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:39:48.541553 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:33923
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:38995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:33923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:48.660434 3873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:48.660755 3873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:48.660799 3873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:48.664965 3873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:48.665045 3873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:48.665064 3873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:48.665120 3873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:48.665153 3873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:48.671020 3873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:38995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:33923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:33923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:48.672247 3873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:48.673393 3873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:48.682482 3882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:48.682462 3881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:48.682467 3884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:48.682855 3873 server_base.cc:1034] running on GCE node
I20260809 03:39:48.683495 3873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:48.684742 3873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:48.685925 3873 hybrid_clock.cc:648] HybridClock initialized: now 1786246788685906 us; error 34 us; skew 500 ppm
Aug 09 03:39:48 dist-test-slave-jq4t krb5kdc[3844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246788, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:48.688817 3873 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:39:48.689730 3873 webserver.cc:460] Webserver started at http://127.1.224.126:35169/ using document root <none> and password file <none>
I20260809 03:39:48.690306 3873 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:48.690367 3873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:48.690543 3873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:48.692663 3873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "c273dce9b5ca478484dc7c7413af69a5"
format_stamp: "Formatted at 2026-08-09 03:39:48 on dist-test-slave-jq4t"
server_key: "90d5b94e07844a94eb006d329b4f42b7"
server_key_iv: "fb2d659c7526b332dccaa8978962479c"
server_key_version: "encryptionkey@0"
I20260809 03:39:48.693192 3873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "c273dce9b5ca478484dc7c7413af69a5"
format_stamp: "Formatted at 2026-08-09 03:39:48 on dist-test-slave-jq4t"
server_key: "90d5b94e07844a94eb006d329b4f42b7"
server_key_iv: "fb2d659c7526b332dccaa8978962479c"
server_key_version: "encryptionkey@0"
I20260809 03:39:48.697098 3873 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:39:48.699687 3894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:48.701047 3873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:48.701184 3873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "c273dce9b5ca478484dc7c7413af69a5"
format_stamp: "Formatted at 2026-08-09 03:39:48 on dist-test-slave-jq4t"
server_key: "90d5b94e07844a94eb006d329b4f42b7"
server_key_iv: "fb2d659c7526b332dccaa8978962479c"
server_key_version: "encryptionkey@0"
I20260809 03:39:48.701294 3873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:48.706100 3873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:48.709514 3873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:48.709702 3873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:48.716874 3873 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:33923
I20260809 03:39:48.716892 3995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:33923 every 8 connection(s)
I20260809 03:39:48.717855 3873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:48.719610 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 3873
I20260809 03:39:48.719727 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:48.720042 1921 external_mini_cluster.cc:1428] Setting key baff93642dae60bec12a4718b165689d
I20260809 03:39:48.721048 3997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:48.725515 4001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:48.721532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35143 (local address 127.1.224.126:33923)
0809 03:39:48.721993 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:48.722009 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:48.722045 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:39:48.722688 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:48.722704 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:48.722986 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:48.723266 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:48.724162 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:48.724167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:48.724179 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:48.724190 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:48.724245 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:48.724264 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:48.724378 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:48.724552 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 03:39:48.724725 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":159,"threads_started":1}
I20260809 03:39:48.727901 3997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5: Bootstrap starting.
I20260809 03:39:48.728578 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:48.730754 3997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:48.731827 3997 log.cc:824] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:48.732812 4001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:48.730949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45993 (local address 127.1.224.126:33923)
0809 03:39:48.731095 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:48.731099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:48.731330 (+ 231us) server_negotiation.cc:407] Connection header received
0809 03:39:48.731443 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:48.731445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:48.731478 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:48.731545 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:48.732330 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:48.732334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:48.732338 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:39:48.732353 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:48.732382 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:39:48.732407 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:48.732542 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:48.732634 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 03:39:48.732678 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:39:48.734091 3997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5: No bootstrap required, opened a new log
I20260809 03:39:48.737716 3997 raft_consensus.cc:348] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c273dce9b5ca478484dc7c7413af69a5" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33923 } }
I20260809 03:39:48.737990 3997 raft_consensus.cc:374] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:39:48.738086 3997 raft_consensus.cc:729] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c273dce9b5ca478484dc7c7413af69a5, State: Initialized, Role: FOLLOWER
I20260809 03:39:48.738555 3997 consensus_queue.cc:260] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c273dce9b5ca478484dc7c7413af69a5" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33923 } }
I20260809 03:39:48.738715 3997 raft_consensus.cc:388] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:39:48.738786 3997 raft_consensus.cc:482] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:39:48.738891 3997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:48.739845 3997 raft_consensus.cc:504] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c273dce9b5ca478484dc7c7413af69a5" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33923 } }
I20260809 03:39:48.740206 3997 leader_election.cc:302] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c273dce9b5ca478484dc7c7413af69a5; no voters:
I20260809 03:39:48.740602 3997 leader_election.cc:288] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:39:48.740701 4016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:48.740967 4016 raft_consensus.cc:686] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [term 1 LEADER]: Becoming Leader. State: Replica: c273dce9b5ca478484dc7c7413af69a5, State: Running, Role: LEADER
I20260809 03:39:48.741322 4016 consensus_queue.cc:237] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c273dce9b5ca478484dc7c7413af69a5" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33923 } }
I20260809 03:39:48.741950 3997 sys_catalog.cc:564] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:48.743167 4018 sys_catalog.cc:455] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c273dce9b5ca478484dc7c7413af69a5. Latest consensus state: current_term: 1 leader_uuid: "c273dce9b5ca478484dc7c7413af69a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c273dce9b5ca478484dc7c7413af69a5" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33923 } } }
I20260809 03:39:48.743309 4018 sys_catalog.cc:458] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:48.743625 4017 sys_catalog.cc:455] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c273dce9b5ca478484dc7c7413af69a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c273dce9b5ca478484dc7c7413af69a5" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 33923 } } }
I20260809 03:39:48.743723 4017 sys_catalog.cc:458] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:48.743925 4026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:48.747491 4026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:48.753911 4026 catalog_manager.cc:1347] Generated new cluster ID: c1aa4f0b95b546f8822e733dfc480d98
I20260809 03:39:48.753993 4026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:48.767577 4026 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:48.768030 4026 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:48.775290 4026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c273dce9b5ca478484dc7c7413af69a5: Generated new TSK 0
I20260809 03:39:48.776221 4026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:39:48.780421 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 3873
2026-08-09T03:39:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1632 ms)
[----------] 18 tests from AuthTokenIssuingTest (29944 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:48 dist-test-slave-jq4t krb5kdc[4053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:48 dist-test-slave-jq4t krb5kdc[4053](info): set up 2 sockets
Aug 09 03:39:48 dist-test-slave-jq4t krb5kdc[4053](info): commencing operation
krb5kdc: starting...
W20260809 03:39:50.035010 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:39:50.171741 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:46161
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:34805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:46161,127.1.224.125:32855,127.1.224.124:37663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:50.291714 4085 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:39:50.292028 4085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:50.292053 4085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:50.292158 4085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:50.296172 4085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:50.296239 4085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:50.296259 4085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:50.296276 4085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:50.296291 4085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:50.301849 4085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:34805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:46161,127.1.224.125:32855,127.1.224.124:37663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:46161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:50.303108 4085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:50.304268 4085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:50.313881 4090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:50.313880 4099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:50.314000 4085 server_base.cc:1034] running on GCE node
W20260809 03:39:50.313910 4093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:50.314851 4085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:50.316053 4085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:50.317215 4085 hybrid_clock.cc:648] HybridClock initialized: now 1786246790317203 us; error 30 us; skew 500 ppm
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:50.320219 4085 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:39:50.321257 4085 webserver.cc:460] Webserver started at http://127.1.224.126:42257/ using document root <none> and password file <none>
I20260809 03:39:50.321827 4085 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:50.321892 4085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:50.322082 4085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:50.324257 4085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "efdb8cad6ea94ddd9e01bb450f743c07"
format_stamp: "Formatted at 2026-08-09 03:39:50 on dist-test-slave-jq4t"
server_key: "00d19f94868f7256cd7190d63b1c91c7"
server_key_iv: "442fd67b6ecc8d2333d179329e4db890"
server_key_version: "encryptionkey@0"
I20260809 03:39:50.324893 4085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "efdb8cad6ea94ddd9e01bb450f743c07"
format_stamp: "Formatted at 2026-08-09 03:39:50 on dist-test-slave-jq4t"
server_key: "00d19f94868f7256cd7190d63b1c91c7"
server_key_iv: "442fd67b6ecc8d2333d179329e4db890"
server_key_version: "encryptionkey@0"
I20260809 03:39:50.328915 4085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:39:50.331743 4108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:50.333117 4085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:39:50.333276 4085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "efdb8cad6ea94ddd9e01bb450f743c07"
format_stamp: "Formatted at 2026-08-09 03:39:50 on dist-test-slave-jq4t"
server_key: "00d19f94868f7256cd7190d63b1c91c7"
server_key_iv: "442fd67b6ecc8d2333d179329e4db890"
server_key_version: "encryptionkey@0"
I20260809 03:39:50.333418 4085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:50.351715 4085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:50.355340 4085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:50.355521 4085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:50.363656 4085 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:46161
I20260809 03:39:50.364032 4191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:46161 every 8 connection(s)
I20260809 03:39:50.364874 4085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:50.367923 4192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:50.370440 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 4085
I20260809 03:39:50.370581 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:50.370958 1921 external_mini_cluster.cc:1428] Setting key 2afbb5beaca5587ce75bbafc1136bbed
I20260809 03:39:50.373059 4192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.224.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.125@KRBTEST.COM" created.
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:50.390203 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.375720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49206 (local address 127.1.224.126:46161)
0809 03:39:50.376194 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.376226 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.376570 (+ 344us) server_negotiation.cc:407] Connection header received
0809 03:39:50.377643 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.377647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.377825 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.378023 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.378945 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.379957 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.381080 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.381544 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.381719 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.384207 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.384214 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.384226 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.384247 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.386414 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.387011 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.387021 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.387025 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.387185 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.387542 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.387549 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.387553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.388182 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.388337 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.388855 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.389234 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":305,"thread_start_us":147,"threads_started":1}
I20260809 03:39:50.390201 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.375720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:46161 (local address 127.0.0.1:49206)
0809 03:39:50.376354 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.376438 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:50.377316 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:50.378020 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:50.378025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:50.378526 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:50.378778 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.378792 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.380083 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.380087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.380959 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.380969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.381707 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.381711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.381833 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.382434 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:50.382451 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:50.384035 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:50.386571 (+ 2536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.386581 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.386591 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.386852 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.387301 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.387303 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.387305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.387436 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.388325 (+ 889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:50.388332 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:50.388549 (+ 217us) client_negotiation.cc:769] Sending connection context
0809 03:39:50.388726 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 03:39:50.389308 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":195,"thread_start_us":167,"threads_started":1}
I20260809 03:39:50.392941 4192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:39:50.393801 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.393507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:32855 (local address 127.0.0.1:33576)
0809 03:39:50.393651 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.393730 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 03:39:50.394104 4112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.224.125:32855: connect: Connection refused (error 111)
I20260809 03:39:50.395051 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.394804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:32855 (local address 127.0.0.1:33584)
0809 03:39:50.394924 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.394970 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 03:39:50.395241 4192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.125:32855: Network error: Client connection negotiation failed: client connection to 127.1.224.125:32855: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.125.keytab.
Entry for principal kudu/127.1.224.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.125.keytab.
I20260809 03:39:50.403769 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.125:32855
--webserver_interface=127.1.224.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.125.keytab
--principal=kudu/127.1.224.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:34805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:46161,127.1.224.125:32855,127.1.224.124:37663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:39:50.444568 4192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } attempt: 1
I20260809 03:39:50.445535 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.445194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:32855 (local address 127.0.0.1:33590)
0809 03:39:50.445372 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.445447 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 03:39:50.446321 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.446090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:32855 (local address 127.0.0.1:33598)
0809 03:39:50.446207 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.446263 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 03:39:50.446455 4192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.125:32855: Network error: Client connection negotiation failed: client connection to 127.1.224.125:32855: connect: Connection refused (error 111)
I20260809 03:39:50.514658 4192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } attempt: 2
I20260809 03:39:50.515707 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.515373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:32855 (local address 127.0.0.1:33602)
0809 03:39:50.515548 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.515618 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 03:39:50.516758 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.516420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:32855 (local address 127.0.0.1:33618)
0809 03:39:50.516635 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.516690 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260809 03:39:50.516911 4192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.125:32855: Network error: Client connection negotiation failed: client connection to 127.1.224.125:32855: connect: Connection refused (error 111)
W20260809 03:39:50.524255 4204 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:39:50.524685 4204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:50.524750 4204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:50.524832 4204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:50.529001 4204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:50.529093 4204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:50.529129 4204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:50.529166 4204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:50.529204 4204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:50.535924 4204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:34805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:46161,127.1.224.125:32855,127.1.224.124:37663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.125.keytab
--principal=kudu/127.1.224.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.125:32855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.224.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:50.537372 4204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:50.538623 4204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:50.548353 4214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:50.548357 4212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:50.548388 4219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:50.548942 4204 server_base.cc:1034] running on GCE node
I20260809 03:39:50.549597 4204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:50.550906 4204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:50.552085 4204 hybrid_clock.cc:648] HybridClock initialized: now 1786246790552070 us; error 33 us; skew 500 ppm
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:50.559099 4204 init.cc:377] Logged in from keytab as kudu/127.1.224.125@KRBTEST.COM (short username kudu)
I20260809 03:39:50.560237 4204 webserver.cc:460] Webserver started at http://127.1.224.125:35427/ using document root <none> and password file <none>
I20260809 03:39:50.560985 4204 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:50.561054 4204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:50.561321 4204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:50.563540 4204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/data/instance:
uuid: "89b6b21335bf45ec8ac46f3bef125ef9"
format_stamp: "Formatted at 2026-08-09 03:39:50 on dist-test-slave-jq4t"
server_key: "7affd8dc2079fb6bb26db276918298a7"
server_key_iv: "3fad29710084387930f0230fefb5fa57"
server_key_version: "encryptionkey@0"
I20260809 03:39:50.564152 4204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/wal/instance:
uuid: "89b6b21335bf45ec8ac46f3bef125ef9"
format_stamp: "Formatted at 2026-08-09 03:39:50 on dist-test-slave-jq4t"
server_key: "7affd8dc2079fb6bb26db276918298a7"
server_key_iv: "3fad29710084387930f0230fefb5fa57"
server_key_version: "encryptionkey@0"
I20260809 03:39:50.568096 4204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 03:39:50.570665 4234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:50.571991 4204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:39:50.572182 4204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/wal
uuid: "89b6b21335bf45ec8ac46f3bef125ef9"
format_stamp: "Formatted at 2026-08-09 03:39:50 on dist-test-slave-jq4t"
server_key: "7affd8dc2079fb6bb26db276918298a7"
server_key_iv: "3fad29710084387930f0230fefb5fa57"
server_key_version: "encryptionkey@0"
I20260809 03:39:50.572304 4204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:50.586294 4204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:50.589692 4204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:50.589917 4204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:50.597111 4204 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.125:32855
I20260809 03:39:50.597404 4331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.125:32855 every 8 connection(s)
I20260809 03:39:50.598184 4204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
I20260809 03:39:50.601387 4333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:50.601854 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 4204
I20260809 03:39:50.601996 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-1/wal/instance
I20260809 03:39:50.602330 1921 external_mini_cluster.cc:1428] Setting key 50d5f2f60a53d1419847985cbba8b28d
I20260809 03:39:50.606256 4333 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:39:50.608089 4192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } attempt: 3
WARNING: no policy specified for kudu/127.1.224.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.124@KRBTEST.COM" created.
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for kudu/127.1.224.125@KRBTEST.COM
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.125@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:50.622524 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.608562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:32855 (local address 127.0.0.1:33628)
0809 03:39:50.608733 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.608748 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:50.608860 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:50.611174 (+ 2314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:50.611178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:50.611385 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:50.611526 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.611535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.612909 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.612924 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.613824 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.613836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.614704 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.614710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.614947 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.615534 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:50.615554 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:50.617555 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:50.620423 (+ 2868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.620427 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.620430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.620747 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.621131 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.621134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.621137 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.621263 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.621992 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:50.621995 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:50.622087 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 03:39:50.622174 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 03:39:50.622342 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":66}
I20260809 03:39:50.622706 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.608526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49216 (local address 127.1.224.126:46161)
0809 03:39:50.608679 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.608684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.609806 (+ 1122us) server_negotiation.cc:407] Connection header received
0809 03:39:50.610976 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.610980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.611043 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.611143 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.612294 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.612921 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.614386 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.614746 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.614866 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.617862 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.617865 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.617866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.617888 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.620464 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.621005 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.621012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.621017 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.621117 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.621495 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.621500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.621505 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.621790 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.621915 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.622368 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.622526 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":61}
I20260809 03:39:50.623819 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.610161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33628 (local address 127.1.224.125:32855)
0809 03:39:50.610494 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.610505 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.610546 (+ 41us) server_negotiation.cc:407] Connection header received
0809 03:39:50.610712 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.610733 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.610997 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.611189 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.611686 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.612796 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.613963 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.614571 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.614728 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.617728 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.617736 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.617750 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.617781 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.620285 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.620879 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.620886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.620887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.621025 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.621378 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.621381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.621383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.621830 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.623147 (+ 1317us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.623269 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.623411 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":210,"thread_start_us":102,"threads_started":1}
I20260809 03:39:50.623816 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.609057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:46161 (local address 127.0.0.1:49216)
0809 03:39:50.609633 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.609675 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:50.610713 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:50.611359 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:50.611365 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:50.611865 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:50.612096 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.612115 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.613053 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.613057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.614273 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.614283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.614987 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.614994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.615085 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.615735 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:50.615752 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:50.617716 (+ 1964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:50.620592 (+ 2876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.620601 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.620613 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.620879 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.621225 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.621228 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.621230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.621377 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.621921 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:50.621929 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:50.622091 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 03:39:50.622276 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 03:39:50.622632 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":62,"thread_start_us":183,"threads_started":1}
I20260809 03:39:50.626035 4333 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:39:50.626096 4192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:39:50.626983 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.626605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57602)
0809 03:39:50.626840 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.626899 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260809 03:39:50.627735 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.627491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57608)
0809 03:39:50.627625 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.627680 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260809 03:39:50.627921 4192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:37663: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.125@KRBTEST.COM for kudu/127.1.224.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.124.keytab.
Entry for principal kudu/127.1.224.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.124.keytab.
I20260809 03:39:50.633584 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.124:37663
--webserver_interface=127.1.224.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.124.keytab
--principal=kudu/127.1.224.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:34805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:46161,127.1.224.125:32855,127.1.224.124:37663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:39:50.635311 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.626706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33632 (local address 127.1.224.125:32855)
0809 03:39:50.627192 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.627196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.627209 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:39:50.627245 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.627248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.627290 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.627390 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.627873 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.628466 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.629380 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.629623 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.629736 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.631911 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.631916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.631922 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.631959 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.633439 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.633920 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.633923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.633924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.634015 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.634354 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.634356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.634357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.634546 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.634623 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.634951 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.635163 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":407}
I20260809 03:39:50.635456 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.626612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:32855 (local address 127.0.0.1:33632)
0809 03:39:50.626735 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.626751 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:50.626830 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:50.627407 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:50.627410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:50.627652 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:50.627757 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.627763 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.628679 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.628685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.629268 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.629277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.629737 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.629741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.629836 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.630461 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:50.630480 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:50.631775 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:50.633558 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.633563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.633565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.633810 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.634117 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.634123 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.634128 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.634247 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.634646 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:50.634652 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:50.634780 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 03:39:50.634899 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 03:39:50.635083 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 03:39:50.636643 4333 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:39:50.637400 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.637143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57614)
0809 03:39:50.637266 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.637346 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 03:39:50.638204 4239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
I20260809 03:39:50.639209 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.638990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57618)
0809 03:39:50.639104 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.639152 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 03:39:50.639376 4333 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:37663: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
I20260809 03:39:50.659087 4192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } attempt: 1
I20260809 03:39:50.659993 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.659667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57632)
0809 03:39:50.659824 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.659893 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 03:39:50.660816 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.660602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57642)
0809 03:39:50.660712 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.660763 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 03:39:50.660941 4192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:37663: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
I20260809 03:39:50.688624 4333 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } attempt: 1
I20260809 03:39:50.689620 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.689287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57644)
0809 03:39:50.689464 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.689530 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 03:39:50.690465 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.690217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57658)
0809 03:39:50.690339 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.690394 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 03:39:50.690604 4333 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:37663: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
I20260809 03:39:50.736133 4192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } attempt: 2
I20260809 03:39:50.737166 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.736813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57670)
0809 03:39:50.736995 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.737077 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 03:39:50.738116 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.737819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57678)
0809 03:39:50.737971 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.738033 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260809 03:39:50.738276 4192 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:37663: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
W20260809 03:39:50.756940 4343 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:39:50.757334 4343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:50.757400 4343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:50.757503 4343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 03:39:50.758767 4333 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } attempt: 2
I20260809 03:39:50.759727 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.759391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57680)
0809 03:39:50.759561 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.759638 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 03:39:50.760701 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.760400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57692)
0809 03:39:50.760577 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.760635 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260809 03:39:50.760836 4333 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:37663: Network error: Client connection negotiation failed: client connection to 127.1.224.124:37663: connect: Connection refused (error 111)
W20260809 03:39:50.762391 4343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:50.762490 4343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:50.762555 4343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:50.762614 4343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:50.762652 4343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:50.768458 4343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:34805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:46161,127.1.224.125:32855,127.1.224.124:37663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.124.keytab
--principal=kudu/127.1.224.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.124:37663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.224.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:50.769762 4343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:50.770893 4343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:50.780165 4358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:50.780174 4362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:50.780153 4356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:50.780428 4343 server_base.cc:1034] running on GCE node
I20260809 03:39:50.781181 4343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:50.782332 4343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:50.783550 4343 hybrid_clock.cc:648] HybridClock initialized: now 1786246790783536 us; error 34 us; skew 500 ppm
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:50.786638 4343 init.cc:377] Logged in from keytab as kudu/127.1.224.124@KRBTEST.COM (short username kudu)
I20260809 03:39:50.787576 4343 webserver.cc:460] Webserver started at http://127.1.224.124:35243/ using document root <none> and password file <none>
I20260809 03:39:50.788134 4343 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:50.788194 4343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:50.788375 4343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:50.790580 4343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/data/instance:
uuid: "e8caaad148c5452cb1b5f89b453cec38"
format_stamp: "Formatted at 2026-08-09 03:39:50 on dist-test-slave-jq4t"
server_key: "9567ecdfc26e8616216ce8edbde07aab"
server_key_iv: "e20a369dd57b316a943cee7b4fb5336c"
server_key_version: "encryptionkey@0"
I20260809 03:39:50.791143 4343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/wal/instance:
uuid: "e8caaad148c5452cb1b5f89b453cec38"
format_stamp: "Formatted at 2026-08-09 03:39:50 on dist-test-slave-jq4t"
server_key: "9567ecdfc26e8616216ce8edbde07aab"
server_key_iv: "e20a369dd57b316a943cee7b4fb5336c"
server_key_version: "encryptionkey@0"
I20260809 03:39:50.795107 4343 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 03:39:50.797750 4374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:50.799104 4343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 03:39:50.799269 4343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/wal
uuid: "e8caaad148c5452cb1b5f89b453cec38"
format_stamp: "Formatted at 2026-08-09 03:39:50 on dist-test-slave-jq4t"
server_key: "9567ecdfc26e8616216ce8edbde07aab"
server_key_iv: "e20a369dd57b316a943cee7b4fb5336c"
server_key_version: "encryptionkey@0"
I20260809 03:39:50.799399 4343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:50.817696 4343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:50.820936 4343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:50.821163 4343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:50.829252 4343 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.124:37663
I20260809 03:39:50.829545 4457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.124:37663 every 8 connection(s)
I20260809 03:39:50.830369 4343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
I20260809 03:39:50.832772 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 4343
I20260809 03:39:50.832909 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786246494448707-1921-0/minicluster-data/master-2/wal/instance
I20260809 03:39:50.833238 1921 external_mini_cluster.cc:1428] Setting key bf4dc6f5e844ac3c0b46c2c797ca5081
I20260809 03:39:50.833770 4461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:50.837452 4192 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } attempt: 3
I20260809 03:39:50.839108 4461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.124@KRBTEST.COM
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for kudu/127.1.224.124@KRBTEST.COM
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.124@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:50.849649 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.837943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57706)
0809 03:39:50.838101 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.838117 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:50.838206 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:50.839931 (+ 1725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:50.839937 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:50.840127 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:50.840223 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.840228 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.841274 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.841280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.842163 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.842180 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.842781 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.842786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.842878 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.843434 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:50.843447 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:50.844904 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:50.847362 (+ 2458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.847379 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.847383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.847699 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.848143 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.848149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.848152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.848259 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.849057 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:50.849060 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:50.849173 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 03:39:50.849290 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 03:39:50.849444 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 03:39:50.851105 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.834783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57702 (local address 127.1.224.124:37663)
0809 03:39:50.835280 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.835296 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.835336 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:39:50.836044 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.836061 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.836390 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.836801 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.837762 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.838989 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.839985 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.840793 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.840979 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.844181 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.844186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.844194 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.844220 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.847233 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.848186 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.848191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.848194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.848273 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.848621 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.848625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.848627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.848941 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.849149 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.850086 (+ 937us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.850264 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":130,"server-negotiator.queue_time_us":348,"thread_start_us":144,"threads_started":1}
I20260809 03:39:50.851114 4466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.838023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57706 (local address 127.1.224.124:37663)
0809 03:39:50.839630 (+ 1607us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.839638 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.839666 (+ 28us) server_negotiation.cc:407] Connection header received
0809 03:39:50.839719 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.839721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.839761 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.839867 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.840346 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.841124 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.842306 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.842640 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.842747 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.845058 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.845065 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.845070 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.845095 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.847222 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.847851 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.847858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.847868 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.848026 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.848363 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.848367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.848368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.848945 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.849072 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.849943 (+ 871us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.850338 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":1539,"spinlock_wait_cycles":2304,"thread_start_us":157,"threads_started":1}
I20260809 03:39:50.852283 4333 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } attempt: 3
I20260809 03:39:50.854063 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.842017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.1.224.126:46161)
0809 03:39:50.842139 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.842142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.842153 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:39:50.842184 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.842186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.842219 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.842311 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.843228 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.843965 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.845261 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.845645 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.845771 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.849047 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.849051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.849053 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.849077 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.850795 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.851413 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.851417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.851419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.851506 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.851818 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.851822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.851823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.852142 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.852251 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.853699 (+ 1448us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.853913 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:39:50.854060 4467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.841159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:46161 (local address 127.0.0.1:49222)
0809 03:39:50.841580 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.841630 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:50.841732 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:50.842452 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:50.842458 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:50.842949 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:50.843086 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.843096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.844085 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.844089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.845132 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.845144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.845803 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.845808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.845959 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.846477 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:50.846489 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:50.848883 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:50.850948 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.850956 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.850961 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.851289 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.851619 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.851622 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.851624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.851709 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.853370 (+ 1661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:50.853375 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:50.853484 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 03:39:50.853627 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 03:39:50.853817 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":114,"threads_started":1}
I20260809 03:39:50.855767 4192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07: Bootstrap starting.
I20260809 03:39:50.857892 4461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:39:50.858889 4192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:50.859937 4192 log.cc:824] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.125@KRBTEST.COM for kudu/127.1.224.124@KRBTEST.COM
I20260809 03:39:50.862427 4192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07: No bootstrap required, opened a new log
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.124@KRBTEST.COM for kudu/127.1.224.125@KRBTEST.COM
I20260809 03:39:50.867395 4192 raft_consensus.cc:348] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } }
I20260809 03:39:50.867686 4192 raft_consensus.cc:729] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efdb8cad6ea94ddd9e01bb450f743c07, State: Initialized, Role: FOLLOWER
I20260809 03:39:50.868072 4192 consensus_queue.cc:260] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } }
I20260809 03:39:50.869489 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.858677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33642 (local address 127.1.224.125:32855)
0809 03:39:50.858809 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.858815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.859023 (+ 208us) server_negotiation.cc:407] Connection header received
0809 03:39:50.859259 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.859262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.859305 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.859355 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.859953 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.861047 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.862079 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.862385 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.862470 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.865289 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.865293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.865296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.865317 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.867329 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.867853 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.867857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.867858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.867955 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.868279 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.868285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.868289 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.868662 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.868746 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.869127 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.869263 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:39:50.869498 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.857652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49234 (local address 127.1.224.126:46161)
0809 03:39:50.857801 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.857805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.857816 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:39:50.857850 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.857852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.857893 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.857965 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.858844 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.859598 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.860746 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.861076 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.861163 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.865214 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.865217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.865220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.865247 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.867213 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.867892 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.867896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.867897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.867988 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.868450 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.868455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.868456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.868697 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.868833 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.869232 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.869357 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 03:39:50.869805 4192 sys_catalog.cc:564] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:50.870049 4467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.858432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:32855 (local address 127.0.0.1:33642)
0809 03:39:50.858922 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.858937 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:50.859132 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:50.859489 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:50.859492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:50.859666 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:50.859795 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.859803 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.861175 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.861178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.861934 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.861945 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.862629 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.862632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.862735 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.863223 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:50.863239 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:50.865136 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:50.867457 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.867460 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.867463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.867721 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.868079 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.868082 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.868084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.868169 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.868826 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:50.868828 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:50.868935 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 03:39:50.869719 (+ 784us) client_negotiation.cc:240] Negotiation successful
0809 03:39:50.869886 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402}
I20260809 03:39:50.870473 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.852940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57714)
0809 03:39:50.853169 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.853184 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:50.853270 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:50.857290 (+ 4020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:50.857294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:50.857506 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:50.857657 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.857664 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.858845 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.858849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.859700 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.859711 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.860330 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.860334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.860424 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.861041 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:50.861057 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:50.862859 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:50.865038 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.865044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.865049 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.865308 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.865681 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.865685 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.865687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.865780 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.869921 (+ 4141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:50.869924 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:50.870072 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 03:39:50.870173 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 03:39:50.870327 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260809 03:39:50.870862 4461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:39:50.871102 4466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.852985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57714 (local address 127.1.224.124:37663)
0809 03:39:50.857015 (+ 4030us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.857022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.857038 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:39:50.857077 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.857080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.857127 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.857795 (+ 668us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.857892 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.858702 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.859840 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.860199 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.860707 (+ 508us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.863044 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.863050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.863056 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.863090 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.864895 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.865433 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.865436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.865437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.865527 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.866017 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.866023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.866026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.866292 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.870734 (+ 4442us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.870808 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.870950 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3911}
I20260809 03:39:50.871462 4473 sys_catalog.cc:455] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } } }
I20260809 03:39:50.871626 4473 sys_catalog.cc:458] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:50.875213 4333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9: Bootstrap starting.
W20260809 03:39:50.876281 4493 catalog_manager.cc:1558] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:39:50.876503 4493 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.124@KRBTEST.COM for kudu/127.1.224.124@KRBTEST.COM
I20260809 03:39:50.879273 4333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:50.880282 4333 log.cc:824] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:50.880720 4466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.871730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57728 (local address 127.1.224.124:37663)
0809 03:39:50.871824 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.871830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.871841 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:39:50.871873 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.871876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.871920 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.871987 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.872519 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.873196 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.874204 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.874485 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.874559 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.877055 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.877059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.877061 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.877084 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.878836 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.879336 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.879340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.879341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.879425 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.879733 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.879735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.879737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.879952 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.880024 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.880357 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.880543 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 03:39:50.881146 4467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.871400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:37663 (local address 127.0.0.1:57728)
0809 03:39:50.871540 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:50.871558 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:50.871646 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:50.872083 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:50.872085 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:50.872266 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:50.872359 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.872366 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.873314 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.873317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.874074 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:50.874089 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.874688 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.874692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:50.874763 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.875301 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:50.875316 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:50.876921 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:50.878956 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.878959 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.878961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.879224 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.879550 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:50.879554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:50.879557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:50.879644 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:50.880086 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:50.880088 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:50.880192 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 03:39:50.880853 (+ 661us) client_negotiation.cc:240] Negotiation successful
0809 03:39:50.881002 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 03:39:50.879753 4085 master_runner.cc:186] Error getting master registration for 127.1.224.125:32855: OK, instance_id { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" instance_seqno: 1786246790595076 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:39:50.883172 4085 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 03:39:50 dist-test-slave-jq4t krb5kdc[4053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.125@KRBTEST.COM
I20260809 03:39:50.886721 4461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38: Bootstrap starting.
I20260809 03:39:50.889990 4461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:50.890515 4333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9: No bootstrap required, opened a new log
I20260809 03:39:50.890950 4461 log.cc:824] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:50.891233 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.872392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33650 (local address 127.1.224.125:32855)
0809 03:39:50.872580 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.872584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.880965 (+ 8381us) server_negotiation.cc:407] Connection header received
0809 03:39:50.881203 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.881206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.881258 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.881307 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.882056 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.882570 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.883386 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.883697 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.883801 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.887151 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.887155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.887158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.887188 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.888930 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.889447 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.889451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.889453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.889554 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.889886 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.889892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.889897 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.890148 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.890337 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.890887 (+ 550us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.891016 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:39:50.894759 4333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } }
I20260809 03:39:50.895080 4333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b6b21335bf45ec8ac46f3bef125ef9, State: Initialized, Role: FOLLOWER
I20260809 03:39:50.895332 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:50.895435 4333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } }
I20260809 03:39:50.895965 4499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } } }
I20260809 03:39:50.896169 4499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:50.896917 4333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:50.896940 4152 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "efdb8cad6ea94ddd9e01bb450f743c07"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49234
I20260809 03:39:50.897324 4152 raft_consensus.cc:482] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 03:39:50.897465 4152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:50.898895 4152 raft_consensus.cc:504] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } }
I20260809 03:39:50.899955 4152 leader_election.cc:288] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [CANDIDATE]: Term 1 election: Requested vote from peers 89b6b21335bf45ec8ac46f3bef125ef9 (127.1.224.125:32855), e8caaad148c5452cb1b5f89b453cec38 (127.1.224.124:37663)
I20260809 03:39:50.900441 4289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "89b6b21335bf45ec8ac46f3bef125ef9"
I20260809 03:39:50.900687 4289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:50.902063 4289 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efdb8cad6ea94ddd9e01bb450f743c07 in term 1.
I20260809 03:39:50.902935 4112 leader_election.cc:302] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89b6b21335bf45ec8ac46f3bef125ef9, efdb8cad6ea94ddd9e01bb450f743c07; no voters:
I20260809 03:39:50.903666 4461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38: No bootstrap required, opened a new log
I20260809 03:39:50.904883 4473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:50.905169 4473 raft_consensus.cc:686] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [term 1 LEADER]: Becoming Leader. State: Replica: efdb8cad6ea94ddd9e01bb450f743c07, State: Running, Role: LEADER
I20260809 03:39:50.905695 4473 consensus_queue.cc:237] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } }
I20260809 03:39:50.911890 4461 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } }
I20260809 03:39:50.912259 4461 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8caaad148c5452cb1b5f89b453cec38, State: Initialized, Role: FOLLOWER
I20260809 03:39:50.908231 4204 master_runner.cc:186] Error getting master registration for 127.1.224.124:37663: OK, instance_id { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" instance_seqno: 1786246790827034 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:39:50.912420 4204 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:39:50.912853 4461 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } }
I20260809 03:39:50.913439 4537 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } } }
I20260809 03:39:50.913643 4537 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:50.914364 4461 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:50.914474 4414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e8caaad148c5452cb1b5f89b453cec38"
I20260809 03:39:50.914732 4414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:50.916131 4414 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efdb8cad6ea94ddd9e01bb450f743c07 in term 1.
I20260809 03:39:50.916936 4533 sys_catalog.cc:455] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efdb8cad6ea94ddd9e01bb450f743c07. Latest consensus state: current_term: 1 leader_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } } }
I20260809 03:39:50.917105 4533 sys_catalog.cc:458] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:50.917567 4546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:50.918606 4546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260809 03:39:50.919833 4522 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:39:50.919924 4522 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:39:50.921442 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.902510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49242 (local address 127.1.224.126:46161)
0809 03:39:50.902659 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.902664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.909505 (+ 6841us) server_negotiation.cc:407] Connection header received
0809 03:39:50.909717 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.909720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.909771 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.909824 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.910954 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.911725 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.912906 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.913252 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.913357 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.917588 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.917592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.917595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.917626 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.919406 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.919945 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.919949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.919951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.920035 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.920335 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.920339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.920341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.920600 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.920715 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.921158 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.921285 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:39:50.931847 4289 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [term 1 FOLLOWER]: Refusing update from remote peer efdb8cad6ea94ddd9e01bb450f743c07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:50.933125 4414 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [term 1 FOLLOWER]: Refusing update from remote peer efdb8cad6ea94ddd9e01bb450f743c07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:50.934031 4533 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:50.934504 4533 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:50.935408 4466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.908834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57736 (local address 127.1.224.124:37663)
0809 03:39:50.908965 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.908970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.924134 (+ 15164us) server_negotiation.cc:407] Connection header received
0809 03:39:50.924296 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.924299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.924347 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.924396 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.925309 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.926075 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.927211 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.927540 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.927607 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.931721 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.931725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.931728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.931756 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.933485 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.933994 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.933998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.934000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.934087 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.934376 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.934380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.934381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.934609 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.934757 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.935120 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.935250 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 03:39:50.937453 4558 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:39:50.937598 4558 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:39:50.940776 4537 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efdb8cad6ea94ddd9e01bb450f743c07. Latest consensus state: current_term: 1 leader_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } } }
I20260809 03:39:50.940922 4537 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:50.944154 4499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efdb8cad6ea94ddd9e01bb450f743c07. Latest consensus state: current_term: 1 leader_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } } }
I20260809 03:39:50.944310 4499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:50.945935 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:50.921330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33658 (local address 127.1.224.125:32855)
0809 03:39:50.921457 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:50.921462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:50.923127 (+ 1665us) server_negotiation.cc:407] Connection header received
0809 03:39:50.923315 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:50.923318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:50.923370 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:50.923422 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:50.924389 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.925117 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.926186 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:50.926497 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:50.926579 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:50.929752 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:50.929756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:50.929759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:50.929787 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:50.936634 (+ 6847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.944330 (+ 7696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.944336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.944338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.944447 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:50.944822 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:50.944826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:50.944828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:50.945099 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:50.945217 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:50.945621 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 03:39:50.945773 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:39:50.949877 4473 sys_catalog.cc:455] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } } }
I20260809 03:39:50.950031 4473 sys_catalog.cc:458] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:50.950172 4560 mvcc.cc:205] Tried to move back new op lower bound from 7316466855649058816 to 7316466855553347584. Current Snapshot: MvccSnapshot[applied={T|T < 7316466855649058816}]
I20260809 03:39:50.951499 4563 mvcc.cc:205] Tried to move back new op lower bound from 7316466855649058816 to 7316466855553347584. Current Snapshot: MvccSnapshot[applied={T|T < 7316466855649058816}]
I20260809 03:39:50.951844 4537 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } } }
I20260809 03:39:50.951960 4537 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:50.953763 4533 sys_catalog.cc:455] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } } }
I20260809 03:39:50.953898 4533 sys_catalog.cc:458] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07 [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:50.955366 4499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efdb8cad6ea94ddd9e01bb450f743c07" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 46161 } } peers { permanent_uuid: "89b6b21335bf45ec8ac46f3bef125ef9" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 32855 } } peers { permanent_uuid: "e8caaad148c5452cb1b5f89b453cec38" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 37663 } } }
I20260809 03:39:50.955492 4499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:50.961254 4546 catalog_manager.cc:1347] Generated new cluster ID: 0867b1a524784fbfabd54dc33b2fc629
I20260809 03:39:50.961344 4546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:50.999547 4546 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:51.001230 4546 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:51.015275 4546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efdb8cad6ea94ddd9e01bb450f743c07: Generated new TSK 0
I20260809 03:39:51.016436 4546 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:39:51.054291 4196 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.050765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49248 (local address 127.1.224.126:46161)
0809 03:39:51.050945 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.050951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.053064 (+ 2113us) server_negotiation.cc:407] Connection header received
0809 03:39:51.053292 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.053296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.053352 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.053406 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.054118 (+ 712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49248: BlockingRecv error: recv got EOF from 127.0.0.1:49248 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260809 03:39:51.055809 4340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.051746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33666 (local address 127.1.224.125:32855)
0809 03:39:51.051870 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.051875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.054834 (+ 2959us) server_negotiation.cc:407] Connection header received
0809 03:39:51.055015 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.055018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.055070 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.055122 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.055678 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33666: BlockingRecv error: recv got EOF from 127.0.0.1:33666 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 03:39:51.057525 4466 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.052419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57748 (local address 127.1.224.124:37663)
0809 03:39:51.052600 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.052605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.056593 (+ 3988us) server_negotiation.cc:407] Connection header received
0809 03:39:51.056772 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.056775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.056827 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.056878 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.057385 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57748: BlockingRecv error: recv got EOF from 127.0.0.1:57748 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 03:39:51.062539 4340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.060869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33668 (local address 127.1.224.125:32855)
0809 03:39:51.060983 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.060988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.061463 (+ 475us) server_negotiation.cc:407] Connection header received
0809 03:39:51.061733 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.061737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.061820 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.061872 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.062411 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33668: BlockingRecv error: recv got EOF from 127.0.0.1:33668 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 03:39:51.068292 4340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.066593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33674 (local address 127.1.224.125:32855)
0809 03:39:51.066709 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.066714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.067285 (+ 571us) server_negotiation.cc:407] Connection header received
0809 03:39:51.067500 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.067504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.067558 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.067617 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.068168 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33674: BlockingRecv error: recv got EOF from 127.0.0.1:33674 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 03:39:51.077986 4340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.076058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33682 (local address 127.1.224.125:32855)
0809 03:39:51.076178 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.076183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.076932 (+ 749us) server_negotiation.cc:407] Connection header received
0809 03:39:51.077138 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.077142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.077212 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.077265 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.077858 (+ 593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33682: BlockingRecv error: recv got EOF from 127.0.0.1:33682 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 03:39:51.086699 4340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.084804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33688 (local address 127.1.224.125:32855)
0809 03:39:51.084968 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.084974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.085603 (+ 629us) server_negotiation.cc:407] Connection header received
0809 03:39:51.085832 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.085835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.085894 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.085949 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.086567 (+ 618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33688: BlockingRecv error: recv got EOF from 127.0.0.1:33688 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260809 03:39:51.099102 4340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.097504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33704 (local address 127.1.224.125:32855)
0809 03:39:51.097632 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.097638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.097952 (+ 314us) server_negotiation.cc:407] Connection header received
0809 03:39:51.098211 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.098217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.098292 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.098396 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.098925 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33704: BlockingRecv error: recv got EOF from 127.0.0.1:33704 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 03:39:51.119145 4340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.117731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33712 (local address 127.1.224.125:32855)
0809 03:39:51.117860 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.117865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.118179 (+ 314us) server_negotiation.cc:407] Connection header received
0809 03:39:51.118318 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.118322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.118389 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.118501 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.119008 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33712: BlockingRecv error: recv got EOF from 127.0.0.1:33712 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 03:39:51.154618 4340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.153454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33714 (local address 127.1.224.125:32855)
0809 03:39:51.153655 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.153660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.153677 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:39:51.153841 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.153844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.153893 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.153967 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.154449 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33714: BlockingRecv error: recv got EOF from 127.0.0.1:33714 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260809 03:39:51.222312 4340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.220930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33730 (local address 127.1.224.125:32855)
0809 03:39:51.221126 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.221131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.221301 (+ 170us) server_negotiation.cc:407] Connection header received
0809 03:39:51.221494 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.221497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.221557 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.221658 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.222165 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33730: BlockingRecv error: recv got EOF from 127.0.0.1:33730 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 03:39:51.353899 4340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.352590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33746 (local address 127.1.224.125:32855)
0809 03:39:51.352782 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.352787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.352933 (+ 146us) server_negotiation.cc:407] Connection header received
0809 03:39:51.353133 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.353136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.353195 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.353289 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.353773 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33746: BlockingRecv error: recv got EOF from 127.0.0.1:33746 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 03:39:51.613579 4340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:51.612309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33754 (local address 127.1.224.125:32855)
0809 03:39:51.612471 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:51.612502 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:51.612603 (+ 101us) server_negotiation.cc:407] Connection header received
0809 03:39:51.612789 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:51.612793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:51.612849 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:51.612941 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:51.613443 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33754: BlockingRecv error: recv got EOF from 127.0.0.1:33754 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 03:39:51.921622 4522 catalog_manager.cc:1259] Loaded cluster ID: 0867b1a524784fbfabd54dc33b2fc629
I20260809 03:39:51.921721 4522 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9: loading cluster ID for follower catalog manager: success
I20260809 03:39:51.923632 4522 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9: acquiring CA information for follower catalog manager: success
I20260809 03:39:51.924984 4522 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 89b6b21335bf45ec8ac46f3bef125ef9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:39:51.939129 4558 catalog_manager.cc:1259] Loaded cluster ID: 0867b1a524784fbfabd54dc33b2fc629
I20260809 03:39:51.939208 4558 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38: loading cluster ID for follower catalog manager: success
I20260809 03:39:51.940903 4558 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38: acquiring CA information for follower catalog manager: success
I20260809 03:39:51.942080 4558 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e8caaad148c5452cb1b5f89b453cec38: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:39:52.132109 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:52.127980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.1.224.125:32855)
0809 03:39:52.128253 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:52.128258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:52.128275 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:39:52.128387 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:52.128390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:52.128570 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:52.128668 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:52.128967 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:52.129523 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:52.130476 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:52.130731 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:52.130842 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:52.131223 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:52.131710 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:52.131816 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:52.131901 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 03:39:52.131963 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":86,"threads_started":1}
I20260809 03:39:52.139618 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:52.135510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57758 (local address 127.1.224.124:37663)
0809 03:39:52.135782 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:52.135789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:52.135807 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:39:52.135915 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:52.135918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:52.136046 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:52.136183 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:52.136425 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:52.137114 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:52.138063 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:52.138292 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:52.138404 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:52.138731 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:52.139184 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:52.139322 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:52.139386 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 03:39:52.139451 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":88,"threads_started":1}
I20260809 03:39:52.146965 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:52.142790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33768 (local address 127.1.224.125:32855)
0809 03:39:52.142940 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:52.142944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:52.143288 (+ 344us) server_negotiation.cc:407] Connection header received
0809 03:39:52.143400 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:52.143403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:52.143461 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:52.143542 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:52.143802 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:52.144534 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:52.145599 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:52.145890 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:52.145996 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:52.146440 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:52.146533 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:52.146617 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:52.146712 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 03:39:52.146766 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:39:52.147018 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:52.142906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57766 (local address 127.1.224.124:37663)
0809 03:39:52.143029 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:52.143034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:52.143263 (+ 229us) server_negotiation.cc:407] Connection header received
0809 03:39:52.143454 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:52.143457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:52.143539 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:52.143611 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:52.143922 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:52.144557 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:52.145683 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:52.146014 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:52.146111 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:52.146476 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:52.146562 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:52.146637 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:52.146800 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 03:39:52.146853 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:39:52.148089 4710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:52.142761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49252 (local address 127.1.224.126:46161)
0809 03:39:52.143102 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:52.143106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:52.143538 (+ 432us) server_negotiation.cc:407] Connection header received
0809 03:39:52.143646 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:52.143648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:52.143825 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:52.143942 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:52.144296 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:52.145083 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:52.146249 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:52.146573 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:52.146672 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:52.147104 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:52.147496 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:52.147610 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:52.147899 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 03:39:52.147966 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":84,"threads_started":1}
I20260809 03:39:52.151127 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 4085
I20260809 03:39:52.168686 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 4204
I20260809 03:39:52.181380 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 4343
2026-08-09T03:39:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3397 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:39:52 dist-test-slave-jq4t krb5kdc[4726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:39:52 dist-test-slave-jq4t krb5kdc[4726](info): set up 2 sockets
Aug 09 03:39:52 dist-test-slave-jq4t krb5kdc[4726](info): commencing operation
krb5kdc: starting...
W20260809 03:39:53.435736 1921 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/test-user.keytab.
Aug 09 03:39:53 dist-test-slave-jq4t krb5kdc[4726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:39:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:39:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
Entry for principal kudu/127.1.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab.
I20260809 03:39:53.569500 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:40305
--webserver_interface=127.1.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:40305,127.1.224.125:38125,127.1.224.124:46855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:39:53.686486 4745 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:39:53.686798 4745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:53.686823 4745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:53.686933 4745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:53.691179 4745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:53.691260 4745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:53.691282 4745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:53.691298 4745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:53.691314 4745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:53.697088 4745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:40305,127.1.224.125:38125,127.1.224.124:46855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.126.keytab
--principal=kudu/127.1.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.126:40305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:53.698307 4745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:53.699400 4745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:53.708374 4759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:53.708623 4745 server_base.cc:1034] running on GCE node
W20260809 03:39:53.708410 4751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:53.708371 4756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:53.709405 4745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:53.710502 4745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:53.711697 4745 hybrid_clock.cc:648] HybridClock initialized: now 1786246793711671 us; error 40 us; skew 500 ppm
Aug 09 03:39:53 dist-test-slave-jq4t krb5kdc[4726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246793, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:53.714689 4745 init.cc:377] Logged in from keytab as kudu/127.1.224.126@KRBTEST.COM (short username kudu)
I20260809 03:39:53.715677 4745 webserver.cc:460] Webserver started at http://127.1.224.126:39941/ using document root <none> and password file <none>
I20260809 03:39:53.716276 4745 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:53.716373 4745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:53.716673 4745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:53.718811 4745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/data/instance:
uuid: "29ea501e1ad04d7199691aff54de517a"
format_stamp: "Formatted at 2026-08-09 03:39:53 on dist-test-slave-jq4t"
server_key: "fd48a644182035246f82c0fc252cd5cc"
server_key_iv: "e3796d2084180abfbdc40c5b7fee71e4"
server_key_version: "encryptionkey@0"
I20260809 03:39:53.719394 4745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/wal/instance:
uuid: "29ea501e1ad04d7199691aff54de517a"
format_stamp: "Formatted at 2026-08-09 03:39:53 on dist-test-slave-jq4t"
server_key: "fd48a644182035246f82c0fc252cd5cc"
server_key_iv: "e3796d2084180abfbdc40c5b7fee71e4"
server_key_version: "encryptionkey@0"
I20260809 03:39:53.723259 4745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:39:53.725837 4771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:53.727090 4745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:39:53.727254 4745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/wal
uuid: "29ea501e1ad04d7199691aff54de517a"
format_stamp: "Formatted at 2026-08-09 03:39:53 on dist-test-slave-jq4t"
server_key: "fd48a644182035246f82c0fc252cd5cc"
server_key_iv: "e3796d2084180abfbdc40c5b7fee71e4"
server_key_version: "encryptionkey@0"
I20260809 03:39:53.727375 4745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:53.745851 4745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:53.749449 4745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:53.749698 4745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:53.757498 4745 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.126:40305
I20260809 03:39:53.757841 4863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.126:40305 every 8 connection(s)
I20260809 03:39:53.758641 4745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/data/info.pb
I20260809 03:39:53.761780 4866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:53.765931 4866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:39:53.767354 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 4745
I20260809 03:39:53.767521 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-0/wal/instance
I20260809 03:39:53.767839 1921 external_mini_cluster.cc:1428] Setting key d7628c6e320a1f0e45a8ead60f06ffe6
Aug 09 03:39:53 dist-test-slave-jq4t krb5kdc[4726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246793, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.224.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.125@KRBTEST.COM" created.
I20260809 03:39:53.783285 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:53.768530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57740 (local address 127.1.224.126:40305)
0809 03:39:53.769104 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:53.769127 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:53.769682 (+ 555us) server_negotiation.cc:407] Connection header received
0809 03:39:53.770959 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:53.770966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:53.771241 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:53.771455 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:53.772241 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:53.773220 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:53.774895 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:53.775602 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:53.775841 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:53.778333 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:53.778340 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:53.778354 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:53.778381 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:53.779981 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:53.780452 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:53.780456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:53.780457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:53.780658 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:53.780979 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:53.780981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:53.780982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:53.781592 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:53.781739 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:53.782234 (+ 495us) server_negotiation.cc:299] Negotiation successful
0809 03:39:53.782446 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":376,"thread_start_us":157,"threads_started":1}
I20260809 03:39:53.783293 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:53.768530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:40305 (local address 127.0.0.1:57740)
0809 03:39:53.769405 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:53.769541 (+ 136us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:53.770605 (+ 1064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:53.771466 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:53.771471 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:53.771820 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:53.772097 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:53.772110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:53.773374 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:53.773381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:53.774719 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:53.774738 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:53.775782 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:53.775785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:53.775976 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:53.776646 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:53.776664 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:53.778176 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:53.780100 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:53.780108 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:53.780116 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:53.780349 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:53.780764 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:53.780767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:53.780768 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:53.780885 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:53.781753 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:53.781760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:53.781966 (+ 206us) client_negotiation.cc:769] Sending connection context
0809 03:39:53.782167 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 03:39:53.782530 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":114,"spinlock_wait_cycles":56064,"thread_start_us":187,"threads_started":1}
I20260809 03:39:53.785976 4866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:39:53.786810 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:53.786540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38125 (local address 127.0.0.1:49952)
0809 03:39:53.786666 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:53.786747 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 03:39:53.787019 4772 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
I20260809 03:39:53.787914 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:53.787693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38125 (local address 127.0.0.1:49960)
0809 03:39:53.787806 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:53.787853 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 03:39:53.788085 4866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.125:38125: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.125.keytab.
Entry for principal kudu/127.1.224.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.125.keytab.
I20260809 03:39:53.798231 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.125:38125
--webserver_interface=127.1.224.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.125.keytab
--principal=kudu/127.1.224.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:40305,127.1.224.125:38125,127.1.224.124:46855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:39:53.837347 4866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } attempt: 1
I20260809 03:39:53.838495 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:53.838065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38125 (local address 127.0.0.1:49974)
0809 03:39:53.838258 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:53.838367 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 03:39:53.839313 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:53.839084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38125 (local address 127.0.0.1:49990)
0809 03:39:53.839199 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:53.839254 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 03:39:53.839458 4866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.125:38125: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
I20260809 03:39:53.907660 4866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } attempt: 2
I20260809 03:39:53.908650 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:53.908268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38125 (local address 127.0.0.1:49996)
0809 03:39:53.908445 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:53.908566 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 03:39:53.909430 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:53.909191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38125 (local address 127.0.0.1:50010)
0809 03:39:53.909309 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:53.909361 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 03:39:53.909577 4866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.125:38125: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
W20260809 03:39:53.919399 4876 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:39:53.919759 4876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:53.919824 4876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:53.919907 4876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:39:53.924045 4876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:53.924134 4876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:53.924175 4876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:53.924232 4876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:53.924271 4876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:53.929870 4876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:40305,127.1.224.125:38125,127.1.224.124:46855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.125.keytab
--principal=kudu/127.1.224.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.125:38125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.224.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:53.931092 4876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:53.932243 4876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:53.941376 4884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:53.941376 4886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:53.941787 4891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:53.941692 4876 server_base.cc:1034] running on GCE node
I20260809 03:39:53.942436 4876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:53.943567 4876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:53.944759 4876 hybrid_clock.cc:648] HybridClock initialized: now 1786246793944739 us; error 34 us; skew 500 ppm
Aug 09 03:39:53 dist-test-slave-jq4t krb5kdc[4726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246793, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:53.947567 4876 init.cc:377] Logged in from keytab as kudu/127.1.224.125@KRBTEST.COM (short username kudu)
I20260809 03:39:53.948529 4876 webserver.cc:460] Webserver started at http://127.1.224.125:36425/ using document root <none> and password file <none>
I20260809 03:39:53.949138 4876 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:53.949230 4876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:53.949489 4876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:53.951606 4876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/data/instance:
uuid: "bb47c07608584f17bbc42afabab1c470"
format_stamp: "Formatted at 2026-08-09 03:39:53 on dist-test-slave-jq4t"
server_key: "df074483e4149e2766890d8d19ba6d1f"
server_key_iv: "b466e2060b25cba6df0bc2efa9f381ee"
server_key_version: "encryptionkey@0"
I20260809 03:39:53.952196 4876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/wal/instance:
uuid: "bb47c07608584f17bbc42afabab1c470"
format_stamp: "Formatted at 2026-08-09 03:39:53 on dist-test-slave-jq4t"
server_key: "df074483e4149e2766890d8d19ba6d1f"
server_key_iv: "b466e2060b25cba6df0bc2efa9f381ee"
server_key_version: "encryptionkey@0"
I20260809 03:39:53.957988 4876 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 03:39:53.961483 4906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:53.963238 4876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:39:53.963407 4876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/wal
uuid: "bb47c07608584f17bbc42afabab1c470"
format_stamp: "Formatted at 2026-08-09 03:39:53 on dist-test-slave-jq4t"
server_key: "df074483e4149e2766890d8d19ba6d1f"
server_key_iv: "b466e2060b25cba6df0bc2efa9f381ee"
server_key_version: "encryptionkey@0"
I20260809 03:39:53.963564 4876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:54.000748 4866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } attempt: 3
I20260809 03:39:54.001689 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.001352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38125 (local address 127.0.0.1:50022)
0809 03:39:54.001511 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.001584 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 03:39:54.002512 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.002259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38125 (local address 127.0.0.1:50028)
0809 03:39:54.002390 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.002445 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 03:39:54.002710 4866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.125:38125: Network error: Client connection negotiation failed: client connection to 127.1.224.125:38125: connect: Connection refused (error 111)
I20260809 03:39:54.002749 4876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:54.006098 4876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:54.006263 4876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:54.013762 4876 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.125:38125
I20260809 03:39:54.014065 5028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.125:38125 every 8 connection(s)
I20260809 03:39:54.014837 4876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/data/info.pb
I20260809 03:39:54.017910 5031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:54.018631 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 4876
I20260809 03:39:54.018816 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-1/wal/instance
I20260809 03:39:54.019177 1921 external_mini_cluster.cc:1428] Setting key f52d6ea9ce3eb40d4ca327a733904735
I20260809 03:39:54.023101 5031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.224.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.224.124@KRBTEST.COM" created.
Aug 09 03:39:54 dist-test-slave-jq4t krb5kdc[4726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246793, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.125@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:54.039392 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.025322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.1.224.126:40305)
0809 03:39:54.025478 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.025482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.026644 (+ 1162us) server_negotiation.cc:407] Connection header received
0809 03:39:54.027618 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.027621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.027703 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.027801 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.029115 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.029666 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.031043 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.031295 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.031420 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.034743 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.034747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.034749 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.034775 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.036976 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.037554 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.037558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.037560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.037649 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.038016 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.038020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.038022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.038231 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.038343 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.039049 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.039198 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:39:54.040262 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.025889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:40305 (local address 127.0.0.1:57744)
0809 03:39:54.026473 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.026517 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:54.027379 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:54.027988 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:54.027995 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:54.028348 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:54.028960 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.028975 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.029811 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.029815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.030914 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.030925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.031508 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.031515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.031747 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.032403 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:54.032438 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:54.034537 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:54.037116 (+ 2579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.037122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.037142 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.037441 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.037750 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.037754 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.037756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.037915 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.038344 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:54.038353 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:54.038672 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 03:39:54.038986 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 03:39:54.039352 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":164,"threads_started":1}
I20260809 03:39:54.041759 5031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.224.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.124.keytab.
Entry for principal kudu/127.1.224.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.124.keytab.
Aug 09 03:39:54 dist-test-slave-jq4t krb5kdc[4726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246793, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.125@KRBTEST.COM for kudu/127.1.224.125@KRBTEST.COM
I20260809 03:39:54.048066 1921 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
/tmp/dist-test-taskfj0rOk/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.124:46855
--webserver_interface=127.1.224.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.124.keytab
--principal=kudu/127.1.224.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.224.84:38843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.224.126:40305,127.1.224.125:38125,127.1.224.124:46855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:39:54.052208 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.042356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38125 (local address 127.0.0.1:50034)
0809 03:39:54.042516 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.042532 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:54.042619 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:54.043204 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:54.043207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:54.043342 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:54.043442 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.043447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.044679 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.044682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.045389 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.045399 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.046075 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.046078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.046151 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.046619 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:54.046653 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:54.048355 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:54.050502 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.050507 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.050511 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.050785 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.051102 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.051105 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.051108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.051214 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.051781 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:54.051783 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:54.051869 (+ 86us) client_negotiation.cc:769] Sending connection context
0809 03:39:54.051972 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 03:39:54.052081 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 03:39:54.052726 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.042498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50034 (local address 127.1.224.125:38125)
0809 03:39:54.042820 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.042836 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.042856 (+ 20us) server_negotiation.cc:407] Connection header received
0809 03:39:54.042893 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.042897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.043053 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.043214 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.043544 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.044567 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.045500 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.045967 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.046060 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.048570 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.048585 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.048606 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.048640 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.050384 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.050927 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.050931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.050933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.051008 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.051312 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.051315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.051316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.051666 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.051814 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.052099 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.052278 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":145,"threads_started":1}
I20260809 03:39:54.054939 5031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:39:54.055708 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.055439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44464)
0809 03:39:54.055566 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.055647 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 03:39:54.055971 4917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
I20260809 03:39:54.057037 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.056825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44476)
0809 03:39:54.056940 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.056985 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 03:39:54.057300 5031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:46855: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
I20260809 03:39:54.106621 5031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } attempt: 1
I20260809 03:39:54.107688 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.107340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44490)
0809 03:39:54.107525 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.107601 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 03:39:54.108637 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.108347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44504)
0809 03:39:54.108516 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.108572 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 03:39:54.108832 5031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:46855: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
I20260809 03:39:54.145884 4866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } attempt: 4
Aug 09 03:39:54 dist-test-slave-jq4t krb5kdc[4726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246793, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for kudu/127.1.224.125@KRBTEST.COM
I20260809 03:39:54.156253 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.146590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38125 (local address 127.0.0.1:50036)
0809 03:39:54.146837 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.146852 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:54.146951 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:54.147345 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:54.147348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:54.147585 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:54.147726 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.147733 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.149050 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.149055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.149931 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.149945 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.150422 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.150428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.150559 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.151141 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:54.151161 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:54.152722 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:54.154708 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.154711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.154712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.154923 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.155257 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.155264 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.155269 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.155365 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.155802 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:54.155804 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:54.155898 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 03:39:54.155987 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 03:39:54.156118 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260809 03:39:54.156379 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.146724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50036 (local address 127.1.224.125:38125)
0809 03:39:54.147102 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.147106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.147119 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:39:54.147158 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.147160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.147200 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.147264 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.147867 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.148920 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.150062 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.150301 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.150388 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.152883 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.152887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.152891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.152926 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.154579 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.155020 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.155023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.155025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.155118 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.155464 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.155467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.155470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.155707 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.155805 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.156070 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.156215 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290}
I20260809 03:39:54.157289 4866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:39:54.158056 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.157785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44514)
0809 03:39:54.157925 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.157983 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 03:39:54.158757 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.158534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44516)
0809 03:39:54.158644 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.158698 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 03:39:54.158895 4866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:46855: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
W20260809 03:39:54.172609 5041 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:39:54.172966 5041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:39:54.173036 5041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:39:54.173115 5041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 03:39:54.177002 5031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } attempt: 2
W20260809 03:39:54.177263 5041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:39:54.177338 5041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:39:54.177366 5041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:39:54.177394 5041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:39:54.177428 5041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:39:54.178013 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.177678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44528)
0809 03:39:54.177852 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.177929 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 03:39:54.178889 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.178643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44540)
0809 03:39:54.178777 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.178826 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 03:39:54.179067 5031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:46855: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
I20260809 03:39:54.183194 5041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.224.84:38843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.224.126:40305,127.1.224.125:38125,127.1.224.124:46855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/krb5kdc/kudu_127.1.224.124.keytab
--principal=kudu/127.1.224.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.224.124:46855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.224.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:39:54.184237 5041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:39:54.185328 5041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:39:54.193722 5058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:54.193717 5056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:39:54.193822 5055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:39:54.193914 5041 server_base.cc:1034] running on GCE node
I20260809 03:39:54.194610 5041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:39:54.195745 5041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:39:54.196934 5041 hybrid_clock.cc:648] HybridClock initialized: now 1786246794196896 us; error 47 us; skew 500 ppm
Aug 09 03:39:54 dist-test-slave-jq4t krb5kdc[4726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246794, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:39:54.199747 5041 init.cc:377] Logged in from keytab as kudu/127.1.224.124@KRBTEST.COM (short username kudu)
I20260809 03:39:54.200675 5041 webserver.cc:460] Webserver started at http://127.1.224.124:37927/ using document root <none> and password file <none>
I20260809 03:39:54.201224 5041 fs_manager.cc:359] Metadata directory not provided
I20260809 03:39:54.201284 5041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:39:54.201469 5041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:39:54.203569 5041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/data/instance:
uuid: "44ff51bef0e148748ce4e2e2ab3b6c36"
format_stamp: "Formatted at 2026-08-09 03:39:54 on dist-test-slave-jq4t"
server_key: "4e1e4fa2465101668f9cbdefb8882b40"
server_key_iv: "7f40b56fdf10a21139df4e39adad78d0"
server_key_version: "encryptionkey@0"
I20260809 03:39:54.204115 5041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/wal/instance:
uuid: "44ff51bef0e148748ce4e2e2ab3b6c36"
format_stamp: "Formatted at 2026-08-09 03:39:54 on dist-test-slave-jq4t"
server_key: "4e1e4fa2465101668f9cbdefb8882b40"
server_key_iv: "7f40b56fdf10a21139df4e39adad78d0"
server_key_version: "encryptionkey@0"
I20260809 03:39:54.208254 5041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 03:39:54.210901 5073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:39:54.212239 5041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:39:54.212347 5041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/data,/tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/wal
uuid: "44ff51bef0e148748ce4e2e2ab3b6c36"
format_stamp: "Formatted at 2026-08-09 03:39:54 on dist-test-slave-jq4t"
server_key: "4e1e4fa2465101668f9cbdefb8882b40"
server_key_iv: "7f40b56fdf10a21139df4e39adad78d0"
server_key_version: "encryptionkey@0"
I20260809 03:39:54.212451 5041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:39:54.218048 4866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } attempt: 1
I20260809 03:39:54.218883 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.218602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44542)
0809 03:39:54.218744 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.218803 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 03:39:54.219640 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.219419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44554)
0809 03:39:54.219542 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.219590 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 03:39:54.219772 4866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.224.124:46855: Network error: Client connection negotiation failed: client connection to 127.1.224.124:46855: connect: Connection refused (error 111)
I20260809 03:39:54.223642 5041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:39:54.226752 5041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:39:54.226955 5041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:39:54.234346 5041 rpc_server.cc:307] RPC server started. Bound to: 127.1.224.124:46855
I20260809 03:39:54.234655 5170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.224.124:46855 every 8 connection(s)
I20260809 03:39:54.235396 5041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/data/info.pb
I20260809 03:39:54.236779 1921 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu as pid 5041
I20260809 03:39:54.236956 1921 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfj0rOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786246494448707-1921-0/minicluster-data/master-2/wal/instance
I20260809 03:39:54.237286 1921 external_mini_cluster.cc:1428] Setting key 643465886c7b2b4ca5b697c592a2016a
I20260809 03:39:54.238766 5171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:39:54.243808 5171 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:39:54 dist-test-slave-jq4t krb5kdc[4726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.124@KRBTEST.COM
Aug 09 03:39:54 dist-test-slave-jq4t krb5kdc[4726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246794, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.124@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
I20260809 03:39:54.253408 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.238764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44566 (local address 127.1.224.124:46855)
0809 03:39:54.239251 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.239271 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.239316 (+ 45us) server_negotiation.cc:407] Connection header received
0809 03:39:54.240209 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.240232 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.240717 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.241089 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.241826 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.243351 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.244281 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.244990 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.245155 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.248250 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.248257 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.248269 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.248300 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.250347 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.250842 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.250847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.250852 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.250965 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.251275 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.251280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.251281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.251764 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.251962 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.252300 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.252630 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":137,"threads_started":1}
I20260809 03:39:54.255406 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.245968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57748 (local address 127.1.224.126:40305)
0809 03:39:54.246105 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.246109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.246513 (+ 404us) server_negotiation.cc:407] Connection header received
0809 03:39:54.246692 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.246695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.246751 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.246839 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.247751 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.248364 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.249334 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.249564 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.249679 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.252061 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.252065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.252067 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.252092 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.253640 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.254118 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.254123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.254125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.254214 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.254494 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.254498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.254500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.254689 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.254799 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.255140 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.255261 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:39:54.255697 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.245920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.126:40305 (local address 127.0.0.1:57748)
0809 03:39:54.246369 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.246413 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:54.246570 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:54.246898 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:54.246904 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:54.247405 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:54.247588 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.247600 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.248544 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.248547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.249194 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.249204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.249680 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.249684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.249816 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.250317 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:54.250329 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:54.251910 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:54.253773 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.253782 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.253786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.254008 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.254312 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.254315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.254317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.254398 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.254799 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:54.254807 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:54.254919 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 03:39:54.255069 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 03:39:54.255517 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":179,"threads_started":1}
I20260809 03:39:54.256822 5171 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:39:54 dist-test-slave-jq4t krb5kdc[4726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.126@KRBTEST.COM
Aug 09 03:39:54 dist-test-slave-jq4t krb5kdc[4726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246794, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.124@KRBTEST.COM for kudu/127.1.224.125@KRBTEST.COM
I20260809 03:39:54.265542 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.255954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57764 (local address 127.1.224.126:40305)
0809 03:39:54.256069 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.256072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.256082 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:39:54.256111 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.256113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.256145 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.256231 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.256909 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.257652 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.258653 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.258860 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.258950 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.261945 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.261950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.261952 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.261980 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.263657 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.264122 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.264126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.264127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.264217 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.264555 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.264558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.264560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.264801 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.265020 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.265224 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.265360 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:39:54.265542 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.257335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.125:38125 (local address 127.0.0.1:50050)
0809 03:39:54.257640 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.257655 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:54.257743 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:54.257976 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:54.257978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:54.258107 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:54.258197 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.258202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.259072 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.259075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.259604 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.259613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.260250 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.260252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.260321 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.260818 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:54.260836 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:54.262198 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:54.263892 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.263894 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.263896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.264121 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.264436 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.264439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.264440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.264591 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.265020 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:54.265023 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:54.265154 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 03:39:54.265267 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 03:39:54.265392 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231}
I20260809 03:39:54.266101 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.257475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.1.224.125:38125)
0809 03:39:54.257626 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.257631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.257704 (+ 73us) server_negotiation.cc:407] Connection header received
0809 03:39:54.257815 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.257819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.257871 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.257952 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.258310 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.258948 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.259724 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.260130 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.260264 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.262319 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.262322 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.262325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.262359 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.263785 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.264224 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.264227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.264228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.264334 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.264692 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.264695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.264696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.264905 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.265006 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.265469 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.265608 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:39:54.266944 5171 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:39:54.270237 5031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } attempt: 3
Aug 09 03:39:54 dist-test-slave-jq4t krb5kdc[4726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.224.125@KRBTEST.COM
Aug 09 03:39:54 dist-test-slave-jq4t krb5kdc[4726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246794, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.124@KRBTEST.COM for kudu/127.1.224.124@KRBTEST.COM
I20260809 03:39:54.276594 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.267568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.1.224.124:46855)
0809 03:39:54.267659 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.267663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.267676 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:39:54.267743 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.267747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.267786 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.267877 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.268229 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.268808 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.269575 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.270583 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.270957 (+ 374us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.272909 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.272912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.272915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.272940 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.274663 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.275073 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.275079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.275084 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.275174 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.275661 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.275666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.275668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.275899 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.276017 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.276267 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.276383 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Aug 09 03:39:54 dist-test-slave-jq4t krb5kdc[4726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246793, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.125@KRBTEST.COM for kudu/127.1.224.124@KRBTEST.COM
I20260809 03:39:54.276899 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.266838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50054 (local address 127.1.224.125:38125)
0809 03:39:54.266970 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.266974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.266986 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:39:54.267124 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.267127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.267168 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.267258 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.267953 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.268443 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.269260 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.269496 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.269616 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.272984 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.272995 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.272997 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.273028 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.274662 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.275460 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.275463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.275465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.275574 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.275878 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.275882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.275884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.276098 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.276356 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.276540 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.276648 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:39:54.277014 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.267486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44576)
0809 03:39:54.267577 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.267591 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:54.267663 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:54.267886 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:54.267889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:54.268058 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:54.268118 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.268124 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.268924 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.268927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.269445 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.269453 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.270761 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.270767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.270901 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.271362 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:54.271379 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:54.272782 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:54.274773 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.274779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.274784 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.274978 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.275272 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.275274 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.275275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.275338 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.276005 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:54.276008 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:54.276095 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 03:39:54.276742 (+ 647us) client_negotiation.cc:240] Negotiation successful
0809 03:39:54.276855 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":4}
I20260809 03:39:54.280622 1921 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:39:54.281018 5171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36: Bootstrap starting.
I20260809 03:39:54.281594 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.270632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44584)
0809 03:39:54.270735 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.270750 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:54.270837 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:54.271689 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:54.271695 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:54.271869 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:54.271943 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.271948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.272807 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.272811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.274633 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.274646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.275167 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.275174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.275269 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.275650 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:54.275668 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:54.277129 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:54.279595 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.279598 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.279601 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.279863 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.280477 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.280480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.280482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.280608 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.281054 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:54.281059 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:54.281173 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 03:39:54.281292 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 03:39:54.281427 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 03:39:54.281632 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.270921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44584 (local address 127.1.224.124:46855)
0809 03:39:54.271491 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.271497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.271509 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:39:54.271536 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.271538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.271568 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.271671 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.272102 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.272699 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.274780 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.275046 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.275151 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.277380 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.277383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.277385 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.277404 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.279446 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.280002 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.280005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.280006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.280093 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.280715 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.280719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.280720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.280934 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.281042 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.281367 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.281481 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":143,"threads_started":1}
I20260809 03:39:54.282064 4819 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "29ea501e1ad04d7199691aff54de517a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57764
I20260809 03:39:54.284040 5171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:54.284601 5031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470: Bootstrap starting.
I20260809 03:39:54.285059 5171 log.cc:824] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:54.286724 5031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:54.286957 4866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } attempt: 2
I20260809 03:39:54.287364 5171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36: No bootstrap required, opened a new log
I20260809 03:39:54.287509 5031 log.cc:824] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:54.289639 5031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470: No bootstrap required, opened a new log
I20260809 03:39:54.291579 5171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } }
I20260809 03:39:54.291867 5171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ff51bef0e148748ce4e2e2ab3b6c36, State: Initialized, Role: FOLLOWER
I20260809 03:39:54.292225 5171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } }
Aug 09 03:39:54 dist-test-slave-jq4t krb5kdc[4726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246793, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.224.126@KRBTEST.COM for kudu/127.1.224.124@KRBTEST.COM
I20260809 03:39:54.292822 5185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } } }
I20260809 03:39:54.293054 5185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:54.293699 5171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:54.293637 5031 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } }
I20260809 03:39:54.293853 5031 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb47c07608584f17bbc42afabab1c470, State: Initialized, Role: FOLLOWER
I20260809 03:39:54.294239 5031 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } }
I20260809 03:39:54.295698 5031 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:54.296627 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.287381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.224.124:46855 (local address 127.0.0.1:44592)
0809 03:39:54.287543 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 03:39:54.287560 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:39:54.287651 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:39:54.288095 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:39:54.288099 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:39:54.288307 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:39:54.288410 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.288416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.289457 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.289460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.290057 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:39:54.290066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.290675 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.290678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:39:54.290768 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.291291 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:39:54.291304 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:39:54.292753 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:39:54.294868 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.294871 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.294873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.295139 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.295458 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:39:54.295463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:39:54.295466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:39:54.295555 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:39:54.296035 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:39:54.296038 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:39:54.296146 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 03:39:54.296250 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 03:39:54.296402 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 03:39:54.297194 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.287613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44592 (local address 127.1.224.124:46855)
0809 03:39:54.287767 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.287771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.287809 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:39:54.287871 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.287874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.287955 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.288050 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.288575 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.289362 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.290176 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.290540 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.290666 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.292913 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.292917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.292920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.292947 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.294737 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.295260 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.295265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.295267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.295354 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.295652 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.295657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.295660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.295883 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.295976 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.296912 (+ 936us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.297052 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:39:54.298677 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } } }
I20260809 03:39:54.299045 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:54.300323 4866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a: Bootstrap starting.
W20260809 03:39:54.300791 5201 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:39:54.300865 5201 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:39:54.298439 5041 master_runner.cc:186] Error getting master registration for 127.1.224.126:40305: OK, instance_id { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" instance_seqno: 1786246793755277 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:39:54.301645 5041 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:39:54.303457 4866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a: Neither blocks nor log segments found. Creating new log.
I20260809 03:39:54.304383 4866 log.cc:824] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a: Log is configured to *not* fsync() on all Append() calls
I20260809 03:39:54.306734 4866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a: No bootstrap required, opened a new log
I20260809 03:39:54.303301 4876 master_runner.cc:186] Error getting master registration for 127.1.224.126:40305: OK, instance_id { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" instance_seqno: 1786246793755277 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:39:54.307380 4876 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 03:39:54.309427 5223 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:39:54.309512 5223 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:39:54.309851 4866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } }
I20260809 03:39:54.310091 4866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29ea501e1ad04d7199691aff54de517a, State: Initialized, Role: FOLLOWER
I20260809 03:39:54.310427 4866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } }
I20260809 03:39:54.310957 5232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } } }
I20260809 03:39:54.311152 5232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:54.311803 4866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:39:54.311852 4819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 03:39:54.311955 4819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:54.313401 4819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } }
I20260809 03:39:54.314447 4819 leader_election.cc:288] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [CANDIDATE]: Term 1 election: Requested vote from peers bb47c07608584f17bbc42afabab1c470 (127.1.224.125:38125), 44ff51bef0e148748ce4e2e2ab3b6c36 (127.1.224.124:46855)
I20260809 03:39:54.315281 4966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "29ea501e1ad04d7199691aff54de517a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bb47c07608584f17bbc42afabab1c470"
I20260809 03:39:54.315474 4966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:54.316847 4966 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29ea501e1ad04d7199691aff54de517a in term 1.
I20260809 03:39:54.316825 5126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "29ea501e1ad04d7199691aff54de517a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36"
I20260809 03:39:54.317103 5126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:39:54.318214 5126 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29ea501e1ad04d7199691aff54de517a in term 1.
I20260809 03:39:54.317338 4772 leader_election.cc:302] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29ea501e1ad04d7199691aff54de517a, bb47c07608584f17bbc42afabab1c470; no voters:
I20260809 03:39:54.321643 5232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:39:54.321861 5232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [term 1 LEADER]: Becoming Leader. State: Replica: 29ea501e1ad04d7199691aff54de517a, State: Running, Role: LEADER
I20260809 03:39:54.322202 5232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } }
I20260809 03:39:54.324069 5232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29ea501e1ad04d7199691aff54de517a. Latest consensus state: current_term: 1 leader_uuid: "29ea501e1ad04d7199691aff54de517a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } } }
I20260809 03:39:54.324186 5232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [sys.catalog]: This master's current role is: LEADER
W20260809 03:39:54.327988 5256 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:39:54.328079 5256 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:39:54.331353 5277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:39:54.337373 5277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:39:54.340051 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.320277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.1.224.125:38125)
0809 03:39:54.320406 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.320411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.321588 (+ 1177us) server_negotiation.cc:407] Connection header received
0809 03:39:54.321689 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.321692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.321745 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.321820 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.322673 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.323338 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.324237 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.324593 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.324678 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.332760 (+ 8082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.332765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.332768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.332798 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.334551 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.337406 (+ 2855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.337410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.337412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.337527 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.338299 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.338303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.338305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.338568 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.338685 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.339744 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.339877 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:39:54.342082 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.320819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57776 (local address 127.1.224.126:40305)
0809 03:39:54.324419 (+ 3600us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.324433 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.324448 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:39:54.324502 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.324505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.324554 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.324647 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.325799 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.326467 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.329225 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.329570 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.329653 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.335116 (+ 5463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.335120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.335123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.335152 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.336893 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.337873 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.337877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.337879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.337969 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.339020 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.339024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.339026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.339249 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.339356 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.341750 (+ 2394us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.341882 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3528}
I20260809 03:39:54.348729 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:54.321005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44598 (local address 127.1.224.124:46855)
0809 03:39:54.321124 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.321128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.338206 (+ 17078us) server_negotiation.cc:407] Connection header received
0809 03:39:54.338400 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.338404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.338456 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.338507 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.339513 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.340290 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.341504 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:54.341849 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:54.341918 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:54.344955 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:39:54.344959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:39:54.344964 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:39:54.344995 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:39:54.346793 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.347286 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.347290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.347292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.347377 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:39:54.347690 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:39:54.347694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:39:54.347696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:39:54.347920 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:39:54.348054 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:54.348411 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 03:39:54.348563 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:39:54.352388 5126 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [term 1 FOLLOWER]: Refusing update from remote peer 29ea501e1ad04d7199691aff54de517a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:54.353762 4966 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [term 1 FOLLOWER]: Refusing update from remote peer 29ea501e1ad04d7199691aff54de517a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:39:54.354161 5273 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [LEADER]: Connected to new peer: Peer: permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:54.354848 5273 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:39:54.360181 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29ea501e1ad04d7199691aff54de517a. Latest consensus state: current_term: 1 leader_uuid: "29ea501e1ad04d7199691aff54de517a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } } }
I20260809 03:39:54.360193 5185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29ea501e1ad04d7199691aff54de517a. Latest consensus state: current_term: 1 leader_uuid: "29ea501e1ad04d7199691aff54de517a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } } }
I20260809 03:39:54.360320 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:54.360319 5185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:54.361323 5273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29ea501e1ad04d7199691aff54de517a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } } }
I20260809 03:39:54.361474 5273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:54.361598 5279 mvcc.cc:205] Tried to move back new op lower bound from 7316466869659779072 to 7316466869547286528. Current Snapshot: MvccSnapshot[applied={T|T < 7316466869659779072}]
I20260809 03:39:54.363670 5185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29ea501e1ad04d7199691aff54de517a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } } }
I20260809 03:39:54.363792 5185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:54.366161 5280 mvcc.cc:205] Tried to move back new op lower bound from 7316466869659779072 to 7316466869547286528. Current Snapshot: MvccSnapshot[applied={T|T < 7316466869659779072}]
I20260809 03:39:54.374176 5277 catalog_manager.cc:1347] Generated new cluster ID: 1c0cb511bfd5414d95ddbc3332b349e9
I20260809 03:39:54.374280 5277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:39:54.391968 5232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29ea501e1ad04d7199691aff54de517a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } } }
I20260809 03:39:54.392154 5232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a [sys.catalog]: This master's current role is: LEADER
I20260809 03:39:54.395123 5187 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29ea501e1ad04d7199691aff54de517a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29ea501e1ad04d7199691aff54de517a" member_type: VOTER last_known_addr { host: "127.1.224.126" port: 40305 } } peers { permanent_uuid: "bb47c07608584f17bbc42afabab1c470" member_type: VOTER last_known_addr { host: "127.1.224.125" port: 38125 } } peers { permanent_uuid: "44ff51bef0e148748ce4e2e2ab3b6c36" member_type: VOTER last_known_addr { host: "127.1.224.124" port: 46855 } } }
I20260809 03:39:54.395287 5187 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:39:54.396140 5285 mvcc.cc:205] Tried to move back new op lower bound from 7316466869659779072 to 7316466869547286528. Current Snapshot: MvccSnapshot[applied={T|T < 7316466869659779072}]
I20260809 03:39:54.425297 5277 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:39:54.426817 5277 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:39:54.452591 5277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29ea501e1ad04d7199691aff54de517a: Generated new TSK 0
I20260809 03:39:54.453595 5277 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:39:54.494606 5181 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:54.490975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44614 (local address 127.1.224.124:46855)
0809 03:39:54.491142 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.491150 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.493210 (+ 2060us) server_negotiation.cc:407] Connection header received
0809 03:39:54.493498 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.493502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.493562 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.493626 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.494444 (+ 818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44614: BlockingRecv error: recv got EOF from 127.0.0.1:44614 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 03:39:54.496582 4869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:54.491795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57778 (local address 127.1.224.126:40305)
0809 03:39:54.491921 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.491926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.495403 (+ 3477us) server_negotiation.cc:407] Connection header received
0809 03:39:54.495616 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.495620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.495679 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.495738 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.496378 (+ 640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57778: BlockingRecv error: recv got EOF from 127.0.0.1:57778 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 03:39:54.498236 5040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:54.492589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50076 (local address 127.1.224.125:38125)
0809 03:39:54.492726 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.492731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.497231 (+ 4500us) server_negotiation.cc:407] Connection header received
0809 03:39:54.497348 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.497352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.497414 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.497533 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.498102 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50076: BlockingRecv error: recv got EOF from 127.0.0.1:50076 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 03:39:54.503347 5040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:54.500832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50078 (local address 127.1.224.125:38125)
0809 03:39:54.500939 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.500943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.501431 (+ 488us) server_negotiation.cc:407] Connection header received
0809 03:39:54.502536 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.502539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.502620 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.502672 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.503228 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50078: BlockingRecv error: recv got EOF from 127.0.0.1:50078 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 03:39:54.510250 5040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:54.507629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50090 (local address 127.1.224.125:38125)
0809 03:39:54.507756 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.507760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.509219 (+ 1459us) server_negotiation.cc:407] Connection header received
0809 03:39:54.509419 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.509422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.509478 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.509532 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.510113 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50090: BlockingRecv error: recv got EOF from 127.0.0.1:50090 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 03:39:54.517498 5040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:54.515185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50094 (local address 127.1.224.125:38125)
0809 03:39:54.515317 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.515323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.516197 (+ 874us) server_negotiation.cc:407] Connection header received
0809 03:39:54.516446 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.516450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.516575 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.516638 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.517368 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50094: BlockingRecv error: recv got EOF from 127.0.0.1:50094 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 03:39:54.526319 5040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:54.524465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50096 (local address 127.1.224.125:38125)
0809 03:39:54.524704 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.524709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.525279 (+ 570us) server_negotiation.cc:407] Connection header received
0809 03:39:54.525488 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.525491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.525551 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.525604 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.526198 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50096: BlockingRecv error: recv got EOF from 127.0.0.1:50096 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
W20260809 03:39:54.539001 5040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:54.537494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50100 (local address 127.1.224.125:38125)
0809 03:39:54.537637 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.537643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.537925 (+ 282us) server_negotiation.cc:407] Connection header received
0809 03:39:54.538123 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.538127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.538190 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.538249 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.538867 (+ 618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50100: BlockingRecv error: recv got EOF from 127.0.0.1:50100 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 03:39:54.558689 5040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:54.557375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50110 (local address 127.1.224.125:38125)
0809 03:39:54.557558 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.557563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.557727 (+ 164us) server_negotiation.cc:407] Connection header received
0809 03:39:54.557907 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.557911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.557970 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.558074 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.558562 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50110: BlockingRecv error: recv got EOF from 127.0.0.1:50110 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 03:39:54.594264 5040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:54.592986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50116 (local address 127.1.224.125:38125)
0809 03:39:54.593152 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.593158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.593348 (+ 190us) server_negotiation.cc:407] Connection header received
0809 03:39:54.593500 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.593503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.593556 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.593642 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.594110 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50116: BlockingRecv error: recv got EOF from 127.0.0.1:50116 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 03:39:54.661825 5040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:54.660710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50126 (local address 127.1.224.125:38125)
0809 03:39:54.660891 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.660895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.660915 (+ 20us) server_negotiation.cc:407] Connection header received
0809 03:39:54.661079 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.661082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.661133 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.661211 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.661681 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50126: BlockingRecv error: recv got EOF from 127.0.0.1:50126 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 03:39:54.793460 5040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:54.792120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50138 (local address 127.1.224.125:38125)
0809 03:39:54.792273 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:54.792279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:54.792461 (+ 182us) server_negotiation.cc:407] Connection header received
0809 03:39:54.792683 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:54.792687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:54.792739 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:54.792836 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:54.793324 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50138: BlockingRecv error: recv got EOF from 127.0.0.1:50138 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 03:39:55.053529 5040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:39:55.052223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50150 (local address 127.1.224.125:38125)
0809 03:39:55.052409 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:55.052417 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:55.052459 (+ 42us) server_negotiation.cc:407] Connection header received
0809 03:39:55.052697 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:55.052701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:55.052762 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:55.052862 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:39:55.053389 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50150: BlockingRecv error: recv got EOF from 127.0.0.1:50150 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 03:39:55.302392 5201 catalog_manager.cc:1259] Loaded cluster ID: 1c0cb511bfd5414d95ddbc3332b349e9
I20260809 03:39:55.302501 5201 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36: loading cluster ID for follower catalog manager: success
I20260809 03:39:55.304327 5201 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36: acquiring CA information for follower catalog manager: success
I20260809 03:39:55.305613 5201 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 44ff51bef0e148748ce4e2e2ab3b6c36: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:39:55.310676 5223 catalog_manager.cc:1259] Loaded cluster ID: 1c0cb511bfd5414d95ddbc3332b349e9
I20260809 03:39:55.310739 5223 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470: loading cluster ID for follower catalog manager: success
I20260809 03:39:55.312538 5223 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470: acquiring CA information for follower catalog manager: success
I20260809 03:39:55.313809 5223 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bb47c07608584f17bbc42afabab1c470: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:39:55.572749 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:55.568263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49688 (local address 127.1.224.125:38125)
0809 03:39:55.568548 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:55.568552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:55.568569 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:39:55.568727 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:55.568730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:55.568875 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:55.569003 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:55.569356 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:55.569982 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:55.571065 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:55.571306 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:55.571400 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:55.571774 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:55.572288 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:55.572389 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:55.572551 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 03:39:55.572614 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":108,"threads_started":1}
I20260809 03:39:55.580089 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:55.576023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33960 (local address 127.1.224.124:46855)
0809 03:39:55.576288 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:55.576293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:55.576308 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:39:55.576526 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:55.576529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:55.576689 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:55.576823 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:55.577072 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:55.577603 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:55.578477 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:55.578717 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:55.578830 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:55.579173 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:55.579631 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:55.579777 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:55.579819 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 03:39:55.579879 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
I20260809 03:39:55.587421 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:55.583263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33966 (local address 127.1.224.124:46855)
0809 03:39:55.583679 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:55.583683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:55.584005 (+ 322us) server_negotiation.cc:407] Connection header received
0809 03:39:55.584174 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:55.584177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:55.584239 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:55.584313 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:55.584540 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:55.585104 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:55.586183 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:55.586514 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:55.586627 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:55.586891 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:55.586986 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:55.587054 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:55.587219 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 03:39:55.587286 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341}
I20260809 03:39:55.587572 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:55.583293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49698 (local address 127.1.224.125:38125)
0809 03:39:55.583415 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:55.583420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:55.583532 (+ 112us) server_negotiation.cc:407] Connection header received
0809 03:39:55.583787 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:55.583791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:55.583852 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:55.583950 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:55.584257 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:55.585054 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:55.586234 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:55.586569 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:55.586659 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:55.587062 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:55.587147 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:55.587196 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:55.587371 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 03:39:55.587432 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 03:39:55.588362 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:39:55.583441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.1.224.126:40305)
0809 03:39:55.583852 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0809 03:39:55.583857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:39:55.583871 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:39:55.583973 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:39:55.583976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:39:55.584156 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:39:55.584280 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:39:55.584720 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:55.585503 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:55.586518 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:39:55.586912 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:39:55.587027 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:39:55.587390 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:39:55.587785 (+ 395us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:39:55.587899 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 03:39:55.588186 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 03:39:55.588244 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":84,"threads_started":1}
I20260809 03:39:55.599611 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 4745
I20260809 03:39:55.619362 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 4876
I20260809 03:39:55.634699 1921 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfj0rOk/build/debug/bin/kudu with pid 5041
2026-08-09T03:39:55Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3455 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6852 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (301166 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST